A Groovy journey to Open Source during COVID-19: npm-groovy-lint

Except for the last part, the article below has been written in 2020, and I didn’t dare to post it before now: with all talented writers and creators on Medium, who the f*** am I to think I could do the same ?

While posting about Salesforce and sfdx-hardis, the open-source initiative we started in Hardis Group, the new company I work in as Salesforce CTO, I found again the draft of this article and now that I understood that anyone should post anything without fearing to be not good enough , I decided to complete it and post it :)

Genesis

In march 2020, I’ve been told by my company that I’ll have to work from home until at least the beginning of July. Meaning a lof of confined time at home. I wondered… how to spend efficiently all the time I won’t spend working or with my girlfriend ?

This story tells how I really discovered the wonderful world of open source contribution, and why you should consider to be part of it

Bored by the lack of easy groovy linting support available via command line, I had started a little personal project one month before: npm-groovy-lint , whose goal was to embed CodeNarc groovy linter, in order to have clean and homogeneous groovy files and Jenkinsfile’s in my professional repositories.

The embedding of CodeNarc using jdeploy was working well, but as a regular user of the great eslint , I still thought some essential things were missing :

• Easy user handling: CodeNarc is a great low level linter package to embed in a java application, like it has already been done with Eclipse and IntelliJ, but the command line mode is in my opinion way too verbose to be usable “out of the box”, according to my loosy experience
• Better performances: At this time, CodeNarc jar and dependencies were loaded at each lint request by the npm-groovy-lint node package, so it could easily lead to a two minutes delay to lint a 1000 lines groovy file
• Automatic formatting: As strange at it can appear, the only maintained Groovy formatter was in IntelliJ, and I wasn’t gonna use another IDE just for a groovy shared library and some Jenkinsfile’s
• Automatic fix: I am lazy… why fixing manually problems that could be fixed by a machine ?
• Integration in my favorite IDE: Visual Studio Code: I want VsCode to handle my Groovy files like it handles JS/TS files: lint while typing, format on save, quick fixes, view documentation…

Step 1: Easy user handling

Here is an example about how to call CodeNarc with command line, with dependency jars to download as prerequisite, and RuleSet files to manually define

I spent a few hours to understand how this work, download dependencies, struggle with classpath, define RuleSet file, etc…

Then I created the initial version of npm-groovy-lint, a simple jdeploy package embedding jars and command line arguments, and the same command was now the following

It’s shorter, but still verbose, so I decided to define npm-groovy-lint own arguments with optionator, and use my own configuration format.

At runtime, the CLI converts them, then generate temporary files, then call CodeNarc using its native format, so now the same call is much shorter (but behind the scene, it is corresponding to the first verbose example)

I now had the choice between verbose XML, verbose HTML or verbose console text results. In 2020, I really like JSON and its easy parsing… so let’s parse CodeNarc XML result files using xml2js, filter and reformat data, and finally return it as formatted JSON in console or in a file !

I was now able to lint my groovy files on demand, using package.json scripts with VsCode NPM Scripts Runner !

Step 2: Performances

But this was long. Too long. Way too long, to wait for one or two minutes to lint a single file, because of the time spent to run java and all dependencies at each request.

How to accelerate that ? In this micro-services era, why not creating a local CodeNarc Http Server, which would be launched once, then call CodeNarc classes at each lint request received, avoiding to reload java libraries ?

As the tool lints groovy, let’s create it in Groovy !

In order to avoid to make the npm package heavier (16Mb already with the embedded jars!) , I chose to use Java Embedded Http Server (examples), that seemed sufficient to manage simple exchanges between localhost and localhost (if you have better lightweight recommendations, please post a comment :) )

Now every call to npm-groovy-lint checked if the server was running :

• If yes: send CodeNarc arguments via Http, then wait for the response
• If not: launch the local CodeNarc Server with a java command, then perform the Http request
• If still not: (problems may happen depending on user config, better be safe than sorry !), call CodeNarc with a java command, but the performances will be the same than without server

With CodeNarc Http Server, to lint the same files, the process elapsed time went from 60 seconds to 5 seconds compared to without CodeNarc Server !

I published the package on NPM, and few days later I noticed that there was already 300 downloads by week, and the first GitHub stars : apparently my work was also interesting others, and like an common Instagram Bimbo or cupcake lover, I was so delighted to see social reactions on my creation !

Step 3: Formatting and fixing

That’s great to be able to collect errors, but each of my linted files had hundreds of them. I started to correct them manually and quickly saw that it was boring and repetitive.

So I started to define fix rules for the most commonly found errors. The pattern is the following:

• Parse CodeNarc errors with regex to extract useful information (error line number, character position, name of the impacted method, etc …
• Apply a javascript rule to fix the error

As the code base was growing, I added test automation with mocha, to be able to perform unit tests on each fix rule, and avoid regressions in new releases. That allowed me to discover excellent Vs Code plugin Test Explorer UI, allowing to run any single test with one click, which is great for debugging !

npm-groovy-lint was born !

Almost 2 years later

Part written in 11/2021

What became my pet project

npm-groovy-lint is now in version 9.0.0 , and a widely used tool with many contributors, and even some github stars !

I also coded a VsCode extension VsCode Groovy Lint, which is also widely used (soon 100000 installs)

I don’t use Groovy anymore, I fell into JS/TS for my Salesforce CI/CD tools, then into Python that I learned to build MegaLinter, my now current biggest open-source Hobby

As I now lack of time to do it, any Groovy addicts wanting to contribute to npm-groovy-lint and vscode-groovy-lint are very welcome !

What I learnt with Open-Source

Don’t keep your work for yourself.

If you created something that is useful for you and was not existing when you looked for it on google, github or stackexchange: publish it, it can be useful to someone else someday !

Don’t think you know. You know nothing. Neither do I.

As technologies evolve, don’t be afraid to evolve with them, even if it means changing all your work habits ! Years ago, I didn’t know about Git, DevOps, CI/CD… that would be so sad if it was still the case !

Don’t be afraid to be criticized, and see it as a way to learn: Contribute, post, share !

The sooner you get into open-source community, the sooner you learn about how to interact with talented strangers all around the world, and the sooner you may become one of them (or for my case, make some people think that you are one of them^^)

I hope the article helped to convince you to dive into Open-Source community if it’s not already the case, undepending if you are 12 years old or 70: there are only good things to win, that you will not learn at your paid job, even if you work with great people !