Enhancing Tuist's `graph` Command with Interactive Graph

RFCs
1

RFC: Enhancing Tuist’s graph Command with Interactive Visualization Using a CDN-Hosted xcode-graph-component

Author: Andy Kolean

GitHub: GitHub

Need/Problem

Tuist’s existing graph command generates static images of a project’s dependency graph. While useful, this approach has significant limitations:

  • Cluttered Views: In large projects, static images become dense and hard to interpret, making it difficult to understand the relationships between various components.

  • Lack of Interactivity: Developers cannot dynamically explore nodes or relationships, limiting their ability to analyze and gain insights into the project’s architecture.

  • Inefficient Debugging: Identifying issues such as cyclic dependencies, redundant links, or bottlenecks is challenging without interactive tools.

  • Limited Reusability: Customization or reuse across projects requires regenerating images, which is not efficient or scalable.

There is a need for an interactive tool that allows developers to explore their project’s dependencies more dynamically and intuitively, improving understanding, debugging capabilities, and overall productivity.

An initial idea and hosted demo of this concept can be found here: XcodeGraphGenerator – A Tool to Visualize Tuist/Xcode Project Dependencies.

Motivation

Understanding and managing project dependencies is critical, especially in large and complex codebases. The limitations of static dependency graphs hinder developers from effectively analyzing and maintaining their projects.

The constraints we aim to address are:

  • Dynamic Exploration: Developers need the ability to interact with the dependency graph, zooming in on areas of interest, and exploring relationships in detail.

  • Scalability: As projects grow, the tools used to understand them must scale accordingly. Static images do not scale well for large projects.

  • Enhanced Debugging: Interactive tools can help quickly identify and resolve issues such as dependency cycles or misconfigurations.

  • Improved Collaboration: Sharing interactive graphs can facilitate better communication among team members, helping everyone understand the project’s structure.

The motivation behind this proposal is to provide developers with a powerful, interactive visualization tool that integrates seamlessly with Tuist, enhancing their ability to manage and understand complex dependency structures.

Detailed Design

Enhance the tuist graph command to generate an interactive dependency graph visualization using a self-contained HTML file that leverages a CDN-hosted Web Component called xcode-graph-component. The component will use D3.js for creating dynamic and interactive data visualizations.

Key aspects:

  • Lightweight HTML Generation: The command generates an HTML file embedding the serialized graph data.
  • CDN-Hosted Component: The HTML file references the xcode-graph-component hosted on a CDN.
  • No Local Server Required: The HTML file can be opened directly in the browser.
  • Interactivity: Developers can zoom, pan, and interact with the graph.

1. Command Interface

  • Default Behavior:
    • tuist graph: Generates a static graph image (current behavior).
  • Interactive Graph Generation:
    • tuist graph --interactive: Generates an interactive graph.
  • HTML Output Option:
    • tuist graph --output html: Exports the interactive graph as an HTML file.

2. Graph Data Serialization in Swift

Utilize Swift’s Codable protocol to serialize the project graph into JSON format suitable for the web component.

3. HTML File Generation

Generate an index.html file containing:

  • Reference to the CDN-hosted xcode-graph-component.

  • Embedded script tag with the serialized graph data.

  • Custom HTML tag for the web component.

4. CDN-Hosted Web Component

  • Development: Create <xcode-graph-component> using D3.js.

  • Hosting: Host on a reliable CDN.

  • Functionality:

  • Reads the embedded graph data.

  • Renders the interactive graph.

  • Handles user interactions.

5. Opening the Graph

  • Write the index.html to a temporary directory.

  • Open the HTML file in the default browser via a system call.

  • No need for a local web server.

Future Enhancements

1. Hot Reloading with --watch Flag

  • Objective: Automatically update the graph when project files change.

  • Implementation:

  • Introduce a local server using SwiftNIO.

  • Implement file-watching and WebSockets for live updates.

  • Benefits:

  • Real-time updates enhance the development experience.

  • Considerations:

  • Optional feature to avoid added complexity for those who don’t need it.

2. Customization Options

  • Allow users to customize visualization (themes, layouts).

  • Enable passing configuration options to the web component.

3. Export Capabilities

  • Export the interactive graph as an image or PDF.

  • Support exporting graph data in different formats.

4. Integration with Other Tools

  • Encourage integration with other Swift or Xcode tools.

  • Create a shared ecosystem leveraging the CDN-hosted component.

5. Accessibility Enhancements

  • Ensure the component is accessible (ARIA roles, keyboard navigation).

  • Improve usability for all users.

Drawbacks

  • Security Concerns: Loading scripts from a CDN in local files may raise security warnings or be blocked in some browser configurations.

  • Offline Access: Requires internet access to load the CDN-hosted component unless provisions are made for offline use.

  • Browser Compatibility: May require polyfills for older browsers to support modern web features.

  • Maintenance: Responsibility to maintain and update the CDN-hosted component.

  • Complexity: Introducing new tools and technologies may add complexity for some users.

Alternatives

1. Using a Local Web Server from the Start

  • Pros:
    • Enables immediate support for features like hot reloading.
  • Cons:
    • Adds complexity by requiring the management of a local server.
    • Introduces dependencies like SwiftNIO.
    • Potential cross-platform issues.

2. Bundling Visualization Libraries into Tuist CLI

  • Pros:
    • All-in-one solution without external dependencies.
  • Cons:
    • Increases the size of the CLI significantly.
    • Complicates maintenance and ties visualization updates to CLI releases.
    • Less flexibility in updating the visualization component independently.

3. Third-Party Tools

  • Option: Export the graph data for use with external visualization tools (e.g., Gephi).
  • Cons:
    • Requires additional setup and learning curve.
    • Less integrated experience.

Adoption Strategy

  • Backward Compatibility: Existing usage of tuist graph remains unchanged; the interactive feature is opt-in.

  • Documentation: Update documentation to include instructions and examples for the new feature.

  • Gradual Rollout: Begin with core functionality and iterate based on user feedback.

  • Community Engagement: Encourage users to try the new feature and provide feedback for improvements.

How We Teach This

  • Terminology: Use clear and consistent terminology (e.g., “interactive graph visualization”).

  • Documentation: Provide comprehensive guides and examples in the official Tuist documentation.

  • Tutorials and Demos: Create tutorials or videos demonstrating the new feature.

  • Integration in Learning Materials: Incorporate the feature into existing learning resources and onboarding materials.

  • Community Support: Engage with the community through forums and discussions to address questions and gather feedback.

Unresolved Questions

1. Versioning of the CDN Module

  • Solution: Use versioned URLs to reference specific versions of the xcode-graph-component to handle breaking changes.

    <script src="https://cdn.example.com/[email protected]"></script>

2. Security Concerns

  • Solution: Ensure the CDN uses HTTPS and consider using Subresource Integrity (SRI) hashes to verify the integrity of the script.

    <script src="https://cdn.example.com/xcode-graph-component.js"
        integrity="sha384-abc123"
        crossorigin="anonymous"></script>

3. Offline Support

  • Solution:
    • Provide an option to bundle the xcode-graph-component locally for offline use.
    • Fallback to generating a static graph image if the CDN is inaccessible.

4. Browser Compatibility

  • Solution:
    • Test the xcode-graph-component across major browsers.
    • Include polyfills where necessary to support older browsers.

Additional Notes

  • Performance Optimization: Ensure the component performs well with large graphs.

  • Security Auditing: Regularly audit the component for security vulnerabilities.

2

Thanks a lot @ajkolean for such a thorough RFC, that’s really appreciated :purple_heart:

I am onboard with the overall design – this will be huge for our existing Tuist users. And who knows, maybe we can make this functionality work outside of Tuist projects as well. I’ve been thinking that eventually we can read the Xcode projects and convert that to our XcodeGraph representation. If we ensure that the interactive graph works just with XcodeGraph, this should make it easier to adopt it for non-Tuist projects down the line.

I have a couple of minor notes about the proposal.

  • Default Behavior: tuist graph: Generates a static graph image (current behavior).

I believe we can change the default to be interactive instead as the default option should be the one that we believe is the best experience for majority of our users.

tuist graph --interactive: Generates an interactive graph.

I’d reuse the --format option instead. Otherwise, users would be able to specify options that don’t make sense together, such as tuist graph --interactive --format json.

We can have tuist graph --format html instead (and as mentioned, the html option would be the default).

We already have --open and --no-open flags to control whether to open the generated file or not after the command finishes.

tuist graph --output html: Exports the interactive graph as an HTML file.

Let’s use the --format option instead

Utilize Swift’s Codable protocol to serialize the project graph into JSON format suitable for the web component.

Yes, we can use the XcodeGraph model that’s Codable already. We recently released a new major version, so we are committed to not introduce breaking changes there.

CDN-Hosted Web Component

I believe this approach is an amazing fit for this usecase and I can’t wait to see this become a reality.

Hot Reloading with --watch Flag

Hot reloading on change of Project.swift might not be the best idea as those files might not always compile. However, I love the idea – it might be better fit to watching the .pbxproj file instead once (and if) we add support for loading the XcodeGraph from those.

Customization Options

We’ll see what makes sense to customize as folks start to use it.

Integration with Other Tools

As I mentioned above, eventually adding support for reading .pbxproj and loading to XcodeGraph will make this component interesting for the whole ecosystem. I’ll write a separate post to capture that idea as that’s not directly related to the interactive graph.

@ajkolean, is there any way we can support you in this work? We’re very interested in this one, so happy to give a hand. @asmit will help on the design side of things.

3

Thanks so much for the detailed feedback and support @marekfort – I really appreciate it! :purple_heart: I’m thrilled that the overall design resonates with you, and I completely agree that making this functionality accessible outside of Tuist projects has exciting potential. The idea of converting Xcode projects to the XcodeGraph representation is particularly intriguing and could open up a lot of opportunities for broader adoption.

Default Behavior and Command Options:
I see your point about changing the default behavior of tuist graph to generate an interactive graph by default. It aligns well with offering the best experience for most users. Using the --format option instead of introducing --interactive is a great suggestion – it keeps the CLI consistent and avoids conflicting combinations of options like --interactive --format json. I’ll adjust the proposal accordingly, so --format html becomes the default behavior, and static images can still be generated with --format png.

Hot Reloading and File Watching:
I wanted to clarify that the hot reloading idea was more of a brainstorming thought about how this feature could evolve in the future. Beyond simply watching .pbxproj files (which, as you pointed out, is a better fit than Project.swift due to potential compilation issues), another concept could involve allowing users to edit the manifest dynamically within the graph. For example, toggling between static and dynamic frameworks or making other adjustments could create a seamless, interactive experience. Of course, these are just exploratory ideas for potential long-term enhancements, but it’s exciting to think about how we could expand this functionality over time.

Customization and Ecosystem Integration:
I agree that customization options can evolve based on user feedback once the feature is in use. I also replied to your other post about .pbxproj integration and left some comments there – it’s such a promising idea, and I’d love to help explore how that could fit into the broader ecosystem as a next step. Capturing that in a dedicated discussion is a great way to set the stage for deeper collaboration on this.

Design and UI Considerations:
I welcome any insight on the design side! My primary focus is on the core functionality right now, and I trust that the UI can be refined as @asmit has time. I’ve seen his base tokens for the system and think they provide a strong foundation to build on when we get to that stage.

Support and Next Steps:
Thank you for offering support! I’ll get started on this soon and will flag any areas where input or collaboration would be valuable along the way. Your enthusiasm for this feature is hugely motivating, and I’m excited to see where we can take it together.

Thanks again for all the thoughtful input and for helping push this idea forward! :rocket:

4

Can’t wait to see this become a reality :partying_face:

Default Behavior and Command Options

Aligned!

Hot Reloading and File Watching and Customization and Ecosystem Integration

I like the ideas! It will be clearer which ones would be feasible and the most useful once we lay out the groundwork.

In terms of the implementation, I believe we can follow roughly this order:

  1. Core implementation: web component + XcodeGraph
  2. Design and UX: next iteration of the implementation based on @asmit’s input
  3. XcodeGraph support of non-Tuist projects
  4. Additional improvements, we’ll see what makes sense.
1 Like