Extension:Graph/Plans

Tracked in Phabricator
Task T334940

This page is a place for WMF staff and volunteers to gather the information needed for the WMF to decide the role it will play in ensuring the needs the Graph extension emerged to serve continue being met.

Progress

Phases to fix Extension:Graph

Phase	Tracking task	Status	Completion date
Phase 0: Invite volunteers to review roadmap		Done	Done
Phase 1: Validate Security of Sandboxing Approach	T222807	In progress	Projected: 22 December 2023
Phase 2: Transition Graph Extension to Vega 5	T165118	In progress
Phase 3: Define maintenance responsibilities and security/incident response
Phase 4: Deploy Graph Extension (Vega 5)
Phase 5: Optimize the Graph Extension's longevity

Roadmap

📍 Phase 0: Invite volunteers to review roadmap

Objective

Ensure volunteers understand and are sufficiently supportive of Phases 1-5 so that we (staff and volunteers) can efficiently make decisions/tradeoffs as they emerge through implementation.

Steps

1. Volunteers and WMF to discuss the proposed roadmap and improve it as needed.

Phase 1: Validate Security of Sandboxing Approach

Objective

Ensure the sandboxed iFrame is sufficiently secure by 1) implementing the Graph Extension using Vega 5 on the Beta Cluster and 2) publishing the documentation and tooling necessary for volunteers and staff to assess its security.

Steps

1. Complete work in progress sandboxing patch to provide the basic infrastructure for future steps.
2. Create a “test box” page, either as an extension or a Special page in core, which allows execution of arbitrary JavaScript inside the infrastructure sandbox.
3. Deploy the “test box” page on the Beta Cluster, with appropriate initial sandbox rules.
4. Pen test the "test box" page to assess its security and refine the sandbox rules iteratively.
5. Perform any relevant static security analysis against updated code prior to deployment.

Phase 2: Transition Graph Extension to Vega 5

Objective

Ready the Graph Extension for Vega 5 by re-implementing Vega 2 features using the Vega 5-compatible syntax so that, going forward, all new graphs will benefit from Vega 5's security, accessibility, and syntax improvements.

Steps

1. Connect Vega 5 to the sandbox infrastructure; implement additional sandbox checks to the Vega 5 loader as necessary.
2. (Re)implement protocol support for Vega 5, based on the Vega 2 implementations while assessing them for security and maintainability.
3. Deploy Vega 5-in-a-sandbox to the Beta Cluster to allow volunteers to perform initial evaluations.
4. Test and evaluate "Vega 5-in-a-sandbox" approach.
5. Conduct additional security testing with Vega5-specific payloads to follow up on the work in Phase 1.
6. Convert the existing Vega 2-to-5 converter to a “2-to-5 tool”, which helps volunteers convert existing Vega 2 specifications to Vega 5 syntax.
7. Implement a message similar to what Pietrasagh described that prompts people to migrate Vega 2-based graphs to Vega 5.

Phase 3: Define maintenance responsibilities and security/incident response

Objective

Define what level of maintenance support volunteers can expect, who will be responsible for providing that support, and how we will respond to security/incident response going forward so that staff can feel confident deploying the extension and volunteers can feel confident depending on it.

Steps

1. Determine who within WMF will be responsible for security/incident response going forward.
2. Define a maintenance and support plan that sets clear expectations about what volunteers can expect from WMF as it relates to the Graph Extension going forward and what team(s) will be responsible for delivering that support. E.g. upgrading to future versions of Vega and other tasks listed in Phase 5.

Phase 4: Deploy Graph Extension (Vega 5) in a Sandboxed iFrame with a restrictive content security policy to production

Objectives

Offer volunteers access to the information and capabilities disabling the Graph Extension has left them without.

Develop the resources (documentation + tooling) necessary to support volunteer porting efforts

Steps

1. In collaboration with volunteers, create “Vega 5 porting” page linking both to the upstream 2-to-5 porting documentation as well as specific information relative to Extension:Graph. E.g. changes to protocols, wiki best practices, related templates and modules, etc.
   1. If appropriate, documentation can also mention how to access the “last Vega 2” version of the graph-to-be-ported from archive.org.
2. With community, port existing Lua modules which generate Vega 2 output to emit Vega 5 output instead.
3. Port existing Vega 2 graphs to Vega 5 and offer tools that help ease this transition
4. Build a more robust catalog of use cases and what Vega features can be relied on

Phase 5: Optimize the Graph Extension's longevity

Objective

Optimize the Graph Extension's longevity and support the community building graphs on Wikimedia projects.

Steps

1. Deprecate, and then remove support for Vega 2 from the codebase, creating categories or linter tags to mark any remaining Vega 2 graphs.
2. Formally transfer Graph Extension maintenance responsibilities to a Wikimedia Foundation engineering team.
3. Create tools and document processes to facilitate routine updates to Vega from upstream for minor/patch/security releases and commit to an appropriate SLA.

FAQ

1. When can we expect to see a timeline for the Roadmap above?
   1. The team expects there to be a timeline to share during the first or second week of November 2023. By this time, we think we will know whether volunteers see any fundamental issues with the roadmap that warrant it being revised.
2. Once re-enabled, will the Graph Extension support pseudo-protocols? Yes. Once the Graph Extension is re-enabled, it will support the protocols already implemented in T335325. Of course, if there are particular ones you think ought to be supported, we'd value you naming them and sharing why on the talk page, as Snævar, Bawolff, Nikki, and Pietrasagh have started doing. Please note: protocol support is a notably fragile part of the Graph Extension. If/when we come to learn this facet of the extension is causing problems, we might need to disable protocol support.
3. How much time – if any – will elapse between the Graph Extension being re-enabled with support for Vega 5 and support for Vega 2 being discontinued? In short, we do not yet know when support for Vega 2 will be discontinued. This is a timeline we (staff + volunteers) will need to define together. With the above said, here is what we are certain about at it relates to Vega 2 and Vega 5 support:
   1. Being able to provide guidance about when Vega 2 support will be discontinued depends on us first learning how the migration to Vega 5 is going.
   2. We do not plan to support Vega 2 graphs indefinitely
4. What can we do to preserve graphs that have not/can not be converted from Vega 2 to Vega 5? Converting graphs of this sort on a server that allows sandboxed rendering exposes security vulnerabilities the current approach is meant to mitigate. More details in T334940#9161842. With the above said, we estimate the graphs that have not/cannot be ported from Vega 2 to Vega 5 to be relatively small and comprised of:
   1. Graphs that use a custom protocol (e.g. Wikimedia API or WDQS) the sandboxed iFrame approach does not support.
   2. Graphs that can be ported, but no one has undertaken the work to do so.
5. Why do we think it's worthwhile to migrate from Vega 2 to Vega 5?
   1. Vega 2 has been superseded by Vega 3, 4, then 5 upstream.  Upstream and third-party documentation exclusively refers to syntax in “Vega version 3.0 and later”, and it is difficult for new contributors to find documentation relevant to Vega 2.  The last upstream release (bugfix or security) of Vega 2.x was in January 2017.  Vega 5 was released in March 2019 and is still under active maintenance and development, with the latest 5.25.0 release in April 2023.
   2. Volunteers have reported issues with Vega 2's accessibility, syntax, and overall functionality, per this 2023 wish.
   3. Vega 5 has made improvements to the library's expression layer that harden it from a security perspective compared to Vega 2.  It is not perfect, but by introducing a parsed expression grammar it offers a more robust foundation for additional security hardening in the future if it proves necessary.
   4. Maintaining multiple versions of Vega concurrently is unsustainable in the long run. The wiki community is taxed in the attempt to independently support software which is not being maintained upstream. Our efforts are best spent working in cooperation with upstream and third-party developers, and to do this we need to be working from the upstream Vega 5 code base.
6. What might be required to migrate graphs from Vega 2 to Vega 5?
   1. Create a converter that would migrate Vega 2-based graphs to be compatible with Vega 5. @Jdlrobson started work on an initial approach in T335048#8794138.  The initial work needs to be restructured slightly to refocus it on being an aid to manual porting, instead of the automatic translator which was its original goal. Note: We estimate this converter currently works for ~80% of graphs, with diminishing returns on additional engineering effort to cover more.  We do not plan on continuing to invest significant additional engineering resources here, but instead to simply repurpose the existing codebase as an aid to manual porting.
   2. Volunteers would need to update <graph> syntax on a case-by-case basis, aided by (1) the ability to run the existing Vega 2 and new Vega 5 specification side-by-side, (2) the partial Vega2-to-5 porting tool which handles 80% of the “obvious” keyword changes and other mechanical conversions, (3) the upstream Vega2 porting guide, and (4) additional documentation or tools which might be created by the wiki community.
   3. Update the limited number of Scribunto templates on-wiki which generate <graph> output in Vega 2 format to instead output Vega 5.  This requires both lua and Vega expertise, but fixes a larger number of Vega 2 uses on wiki at once.

Proposal

To safely restore access to the information and capabilities disabling the Graph Extension has left people without and promote the volunteer+staff collaboration needed to do so, we (the WMF) are committing to:

Re-enable the Graph Extension in a sandboxed iFrame with a restrictive content security policy.

1. Once the Graph Extension is reenabled, it will continue to work with Vega 2 for a yet-to-be defined period of time. Note: we'll need to define this window together.
   1. After that "yet-to-be defined period of time," Vega 2 support will be discontinued and use of the Graph Extension will require volunteers to make graphs with Vega 5.
   2. As soon as possible, make the sandboxed Graph extension available on the beta cluster for testing. See: T346292.
2. Investigate the viability of adding logging to increase our awareness of instances where people are exploiting the security vulnerabilities inherent with restoring support for Vega on our platform. See T346414.
3. Publish the technical documentation needed for developers across the Movement to understand how we implemented the sandboxed CSP approach
4. Publish a clear timeline for when you all can expect all of the above to happen
5. Note: exploratory work to redeploy the Graph Extension in a sandboxed iframe has started. See T222807.
6. Share regular updates about the progress we're making on the commitments named above on Phabricator and MediaWiki.
7. Support volunteers with code and processes that will ease the transition from Vega 2 to Vega 5 when the time for this transition comes.

In support of the above, we'd need to depend on ya'll (volunteers) to:

1. Spread awareness of this proposal and the updates that will come as we start implementation, assuming this proposal moves forward.
2. Manually migrate some proportion of Vega 2-based graphs to be compatible with Vega 5. See the "questions 5 and 6 in the FAQ section".
3. Potentially, fix/port graphs that attempted to fetch live data using methods that the sandboxing approach inhibits.
   1. Note: the need for the above will become clear once we decide on whether we will restore the pseudo-protocols that were used to fetch data live from the action API, the REST API, WDQS etc, and the precise sandbox parameters we select (domains/ports/http methods allowed). This decision will be made in T346291.

Research

The research that informed the the proposal for safely and securely restoring access to the information and capabilities disabling the Graph Extension has left people without can be found here .