ClojureScript libraries vs. apps

Let's start with a couple of quick definitions:

1. Library: Code that is consumed by other software projects; whether internally, externally, or both.
2. Application: Code that users interact with directly. Often consumes libraries to accomplish what it does.

There isn't always a bright line between app and library code, but let's pretend like there is for now. We'll return to blurry app/library projects later.

⚖️

Assumptions

Second, some assumptions I'm making:

1. The tools-deps approach is the future of Clojure(Script) and we should start with that, all other things being equal.
2. We want to keep things Simple™ and try adopting small, focused tools that do one thing and do it well before we examine more complex tools that attempt to solve multiple problems.

If you disagree with either of those assumptions, then YMMV on how useful you find the rest of this post.

🧳

Apps

For ClojureScript applications (e.g. a re-frame web app), I want to recommend starting with the new :target :bundle feature. But if you read the rest of this blog post, you'll see why I have to recommend shadow-cljs instead for now. UPDATE: Not anymore! I wrote target-bundle-libs to help make :target :bundle work better for libraries.

However, if you do want to use :target :bundle, I'm going to recommend that you use it in a slightly different way than the official docs recommend. This is to make things more consistent between app and library code and allow the two to be used more interchangably. More on all that at the end.

👩🏿‍💻

Libraries

If you don't need to depend on any npm JS libraries in your ClojureScript library, then you can ignore all this. Your CLJS library will just work in whatever apps you use it in and with whatever tools you want to use.

But if you do need to consume one or more JS libraries from npmjs.com in your CLJS library, read on...

This is where things get tricky. And this is why I decided to write this post in the first place. You can't use :target :bundle in libraries in the way it's shown in the official docs on clojurescript.org. From what I can gather, you're supposed to use the under-documented deps.cljs feature for libraries when consumed by :target :bundle apps. That seems unfortunately inconsistent to me, but I am assured that I am wrong and just making things more complicated for everyone. Hey, it wouldn't be the first time. :)

😉

Case study

Alright, let's give this :target :bundle app with deps.cljs libraries thing a try. And then we'll talk about how we feel about it vs. just using shadow-cljs everywhere (or something else entirely?).

The app

For the app we'll generate a basic re-frame web app that uses :target :bundle. You can find that here: https://github.com/cap10morgan/re-frame-target-bundle-example

The library

We will likewise make a very basic CLJS library that wraps a JS library from npmjs.com. Let's choose... the pad library. You can find the CLJS wrapper lib here: https://github.com/cap10morgan/cljs-npm-js-example-lib

Some things to notice:

1. The library specifies the npm deps (pad) that it references from its CLJS code in src/deps.cljs. That's important because it's the only way the CLJS compiler (whether you're building your library for e.g. running its tests or a downstream consumer of your library is building their entire project) can install and find those deps.
   1. NOTE: Do not confuse this file with the similiarly-named but very different deps.edn. src/deps.cljs is for specifying foreign (i.e. not Google Closure compatible) JS dependencies. deps.edn, by contrast, lives in your project root (not src/) and specifies your deps, aliases, paths, etc. for your entire tools-deps project for both CLJ and CLJS.
2. The library still has a package.json file that specifies JS deps that are not referenced from the CLJS code but are nonetheless needed by the project. For example: test runners like karma.
3. It still uses :target :bundle in the test-build.edn file, along with some older CLJS compiler opts :install-deps true and :npm-deps.
4. :npm-deps is set to an empty map {} in test-build.edn because the compiler (as of CLJS 1.10.773) crashes if it isn't set (it seems to default to a boolean value; probably false) and you have :install-deps true. We don't want to redundantly specify the deps here (they're in src/deps.cljs) but setting it to an empty map seems to work for now. That crash when it isn't set is likely just a bug, and fixing it might go a long way to making me feel better overall about this approach.
5. There's no reason you couldn't move your CLJS-referenced JS deps in your application projects from package.json to src/deps.cljs and use this same trick / hack to make them consistent with your libraries.
6. This feels brittle and icky (IMHO). I would much rather the documented :target :bundle approach of specifying all JS deps in package.json also worked for libraries with JS deps. But it doesn't (as of CLJS 1.10.773). And surprisingly (to me), the CLJS developers have no desire to change that.

So what is this deps.cljs file anyway?

Good question! As far as I can tell, support for it in some form was introduced in this commit in 2012 (2012!). But it has been enhanced over time (e.g. this commit).

Essentially it is a way to encode foreign-libs JS dependencies in a way that is accessible to the CLJS compiler and on the classpath so it gets bundled into JARs, etc. That's what allows it to be used to lookup, (optionally) install, and bundle transitive foreign libs. It is primarily docuemented here: https://clojurescript.org/reference/packaging-foreign-deps

Since it contains foreign-libs, the (almost; see below) full reference documentation on what it can include is here: https://clojurescript.org/reference/compiler-options#foreign-libs

...but with an important addendum! Since the :npm-deps compiler option generates :foreign-libs entries under the hood, support for using that in deps.cljs both in the current project and transitively in its dependencies was added in 2017.

So that means you should also consult these docs for the full picture of what can go in deps.cljs: https://clojurescript.org/reference/compiler-options#npm-deps

Did you see the warning in the :npm-deps docs? It says:

The :npm-deps feature is in alpha status for optimized builds. When applying Closure optimizations, NPM dependencies are passed through Closure Compiler and not all NPM libraries contain Closure-compatible code.

A reliable alternative can be found with ClojureScript with Webpack.

...unless you're building a library, in which case you have to keep using :npm-deps, deps.cljs, etc. (for now?).

That warning also only applies to direct :npm-deps, not ones you specify in deps.cljs, apparently. I couldn't find that documented anywhere except this excellent blog post by Herny Widd that I highly recommend as additional reading on modern CLJS library development.

😅

Blurry app/library code

Sometimes you have codebases that are both libraries and apps. That is, they can be used directly by a user or they can be consumed by a downstream application as a library. This isn't well-supported by the official :target :bundle documentated approach currently.

You could, of course, split these codebases up into separate library and app repos. But if you don't want to do that, then the best approach for now seems to be using the "library" approach of putting all CLJS-referenced JS deps in src/deps.cljs and turning on :install-deps true with :npm-deps {} in your compiler opts.

And really, it might just be best for now to structure all of your tools-deps + :target :bundle CLJS projects that way. At least then they're consistent.

🕵🏽‍♀️

Is there a better way?

If, like me, you don't love this, my first recommendation would be to just use shadow-cljs. It will eliminate this issue for you and allow you to structure dependencies for apps, libraries, and app/libraries consistently. And you can use shadow-cljs with tools-deps! So it's almost certainly the best solution for now.

But I also want to see :target :bundle w/o shadow-cljs become a more viable option for advanced CLJS development. So I want to explore how to make this more consistent and less brittle / hacky feeling. Ideally I'd like to find a way to specify JS deps in package.json (only) everywhere and have applications consuming libraries that do this find their JS deps and build correctly.

Some ideas I have for that are:

1. Auto-generate the src/deps.cljs from package.json and some inspection of what JS symbols are referenced from the CLJS code. Or maybe scratch the inspection part and just specify every non-dev dependency. I have no idea how difficult / viable that would be.
2. At a basic level, all you really need is for all JS deps (direct and transitive) to end up in node_modules when your :bundle-cmd executes. So this could also be accomplished by a tool that finds all of those and installs them. After all, :target :bundle itself doesn't directly handle JS dep installation, but expects you to do that w/ the usual npm / yarn tools. Something like a cljs-npm tool that found and installed every transitive JS dep, perhaps? Or something that plugged into npm / yarn and did something similar? Again I have no idea how difficult / viable that would be. (I do worry that package.json files might not typically get bundled into JARs.)
3. Maybe try writing an NPM module that can wrap webpack and be run as the :bundle-cmd and figure all this out w/ a tree of :target :bundle dependencies. That still doesn't solve the problem of how to look up the transitive foreign JS deps, though (I don't think).

If I make any progress on these or any other approaches to this I'll post a "part 2" follow up.

UPDATE: I have published part 2 where I introduce target-bundle-libs.

This deprecates all of the advice below. Now I recommend using target-bundle-libs!

✌🏻

OK but what should I do today, in simplest terms?

Just use target-bundle-libs!

This section originally said (before I wrote target-bundle-libs):

The TL;DR version of my recommendations for now are:

1. Just use shadow-cljs for all of your CLJS development.

OR... if you really want to use tools-deps and :target :bundle today:

1. Specify all of your CLJS-referenced NPM JS libs in src/deps.cljs inside an :npm-deps map (see here for an example).
2. Specify any other JS deps that aren't referenced from your CLJS code in package.json.
3. Add these two key-values to any / all of your build.edn files or anywhere else you specify compiler options: {:install-deps true, :npm-deps {}}.

👋🏻