Exploring the latest features of Skott: road to V1

Hello everyone, I hope you're doing well.

Following the series of My journey of building Skott, an open-source Node.js library, I wanted to share with you a quick update about the latest features added. Implementing these features will be the opportunity for me to share with you what it took to build them.

For those who are not familiar with the tool, I suggest you to quickly check my first article introducing Skott

Summary

• Web application (v0.11.0)
• Incremental comparison (v0.12.0)
• Detecting unused third-party dependencies (v0.13.0)

Web application (introduced in v0.11.0)

For its first versions, Skott's visualization was only rendered in our beloved command line interface which is cool but not really ideal to represent real-world graph structures.

Since v0.11.0, Skott now embeds the new "webapp" display mode, which is the new default display mode.

$ skott --displayMode=webapp

If you want third-party dependencies (npm) and built-in dependencies (Node.js) to be rendered in the app, don't forget to provide the flags to the CLI, otherwise they won't be collected nor displayed:

$ skott --displayMode=webapp --trackThirdPartyDependencies --trackBuiltinDependencies

Once the graph generated, the web application is automatically opened in your default browser on a free port:

The application renders a 2D network in which files are represented by the nodes of the network and the links between these files are represented by directed edges.

Within the left sidebar, few things are displayed:

• Some statistics are exposed (number of files, circular dependencies, etc)
• Some visualization options can be toggled on/off to highlight or make appear additional nodes/edges representing third-party or built-in dependencies.

To make the search of specific files easier when processing large graphs, a global search was recently introduced.

By using the CMD+K/CTRL+K command, files can be searched and then focused on:

Thanks to @bam-charlesbo for suggesting new features about the webapp.

Incremental graph processing (introduced in v0.12.0)

Computing graph is expensive as it includes a static analysis on each file of the project mainly involving parsing and AST walking. Depending on the language, some parsers are faster than others, for instance JavaScript parsers are naturally faster than TypeScript ones as TypeScript embeds a lot more of information encoded in the language at the type-level.

Introducing an incremental comparison

Even if Skott can analyze thousands of files in a matter of few seconds, performance always matter for a better DX. So here is a first step in order to make it nicer!

In the past, I reproduced a very minimalist implementation of the Affected/Incremental pattern that most of the monorepo tools embed natively allowing to gain a lot of performance on heavy project graphs.

If you're interested in knowing more about how it works under the hood, you can directly refer to this article.

Because most of the time project graphs do not entirely change (think of few files changes per commit), using incremental comparison would allow us to heavily benefit from caching.

From v0.12.0, a first version of an incremental comparison can done by providing the --incremental argument from the CLI. A folder ".skott" will be generated at the same location the command was run.

$ skott --incremental 

After re-running the same command, you should be able to see the difference between the time took for the analysis without the cache. Note that difference may seem slim if the project doesn't contain much files.

Note that because resolving paths from a cache involves many edge cases, incremental mode is not turned on by default yet, but will be once the feature covers most of the cases I have in mind. Most of the time using it works well though, so don't hesitate to abuse the feature and open issues if ever you encounter some problems while enabling the incremental feature to make it more stable.

Unused dependencies (introduced in v0.13.0)

The unused dependencies feature will try to cover as much use cases as possible over time. For now what have been introduced in v0.13.0 is the detection of unused npm production dependencies. Note that only production code is analyzed as of now, so if you're using production dependencies in test files, Skott will report them as unused.

$ skott --showUnusedDependencies --trackThirdPartyDependencies --displayMode=raw

--trackThirdPartyDependencies is required for unused npm dependencies to be found.

Thanks to @ild0tt0re for that feature request.

Conclusion

That's it for the latest updates about Skott :) As promised in the latest blog posts of this series, the next one will be about how I leveraged Test-Driven Development and Dependency Injection to confidently develop the whole chain of file exploring, parsing and analysis

Also, here is the repository if you want to know more.

Thanks for reading, see you next time!