Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replacing the way Typegraphs are generated #430

Open
finanalyst opened this issue Jan 12, 2025 · 5 comments
Open

replacing the way Typegraphs are generated #430

finanalyst opened this issue Jan 12, 2025 · 5 comments

Comments

@finanalyst
Copy link
Collaborator

@coke @dontlaugh tl;dr asking for ideas

Currently the build process (and by consequence in particular the way typegraphs are generated) is suboptimal.

Currently (using Collection), changes to sources (the rakudoc files in the Raku/docs/doc/ directory) are detected and new ones are re-cached (good).

All cached sources are re-rendered to HTML (bad). In Elucid8 (the Collection re-write), a record is kept of when html files are generated, and only those that are older than sources are re-rendered. (good). So on a rebuild, only new files are generated.

Currently, pre-generated typegraphs svgs are attached to documentation by matching the name of the typegraph svg to the name of the source. Not all Types have svgs. I'm not sure whether the opposite is true.

Typegraph svgs are independently generated in three stages

  1. A definition file that describes the relationship between classes is created (this was done years ago and has not been recreated)
  2. The definition file is used to generate GraphViz definitions of directed graphs between classes, one definition for each class
  3. GraphViz is used on the directed graphs to generate svg files, one per class (type)

Since the first step hasnt changed for years, nor should the last, but it gets done every build. This is a waste of time and what I would like to change now.

I dont know how the definition file was generated, but it should be done programaticallly. Is anyone sure that the type definitions have not been modified?

Coleman and I have tried to ensure a single build rather than re-generating the svgs each time the htmls are generated. But its been difficult to hook them together. Also, it is not easy to arrange for the Table of Contents to have a reference to the image.

We could adopt other strategies:
a) keep the current situation. Have a pool of svgs and connect the svg to the html at the html build time. I don't like this idea as it depends on the software to keep two pools of files in sync with each other based on file names.
b) have a pool of svgs in some fixed directory, but include a link to the image in a normal way.
advantages:

  • The because the image is a part of the flow of the source, it can be incorporated in the Table of contents
  • when a source is translated, the translator can see there is more material. If the attachment is separate from the build, a translator will not see it.
    disadvantage:
  • each type source will need changing, but this can be done programatically
    c) have the dot digraph definition in the body of the source, and arrange for it to be generated with the html. (The custom block to do this is already a part of both Collection and Elucid8)
    advantages (in addition to the above):
  • gradually the parameters of the rendition from dot to svg (or png) can be tweaked so that the diagram can be adapted to dark and light themes. Currently, the diagrams are not so adaptable.
    disadvantage:
  • the dot description needs to be added to each Type source

I think the source of the documentation and the source of the graphs should be united, whether in svg form, dot form, or a link to an image.

This will need a change to the rakudoc source, but I think this can be done programatically relatively easily.

However there is a cost / benefit to each of the strategies.

Any thoughts?

@dontlaugh
Copy link
Collaborator

the dot description needs to be added to each Type source

I'm wondering how this would look, potentially. Can you paste an example, even a speculative one.

@finanalyst
Copy link
Collaborator Author

@dontlaugh I have this for type/AST.rakudoc. Here's the rendered version.
The RakuDoc source is below (its about the shortest type file in docs). It was generated by
generating the dot file, copying its contents in between the Graphviz bits, at the end of the existing source. (I also stripped out some styling instructions for clarity below, but they dont affect the rendering much and could be left in).

=begin pod :kind("Type") :subkind("class") :category("basic")

=TITLE class AST

=SUBTITLE Abstract representation of a piece of source code

    class AST { }

An C<AST> or I<Abstract Syntax Tree> is a partially processed representation
of a program.  ASTs are return values of the C<quasi> quoting construct,
and are typically used within macros to generate code that is inserted
in the calling location of the macro.

There is no API defined for ASTs yet.  Hopefully that will emerge as
part of the work on macros.

=begin Graphviz :caption<Class relation diagram>
digraph "raku-type-graph" {
    rankdir=BT;
    splines=polyline;
    overlap=false;

    // Types
    "AST" [href="/type/AST"];
    "Mu" [href="/type/Mu"];
    "Any" [href="/type/Any"];

    // Superclasses
    "AST" -> "Any";
    "Any" -> "Mu";
}
=end Graphviz

=end pod

@dontlaugh
Copy link
Collaborator

That does look pretty good for smaller graphs. I say go for it.

It will also let us experiment with other diagram DSLs. I've used Mermaid way more than dot+Graphviz.

We still need to solve the problem of keeping these definitioned sync'd with upstream Rakudo. Relying on the fact that most types don't change that often is no good. We should be able to derive these class/type relationships from Rakudo source code, but I'm not aware of any script that does that, currently.

@coke
Copy link
Contributor

coke commented Jan 13, 2025

I can work on the derivation from Rakudo source to a (e.g.) file we can check in - we can regen them as needed (occasionally) and the the dates can be used in the build system to determine if a page needs to be regen'd.

@finanalyst
Copy link
Collaborator Author

@coke not sure if you agree to including the graph definitions within the rakudoc sources or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants