The frustrations of opensource

I spend a lot of time “re-inventing the wheel”, usually due to being blocked or frustrated by the wheel I’m trying to use.

Frustrations and roadblocks that occur when dealing with opensource software can often get in the way of writing software, and when that’s your job, you can’t just wait a few months for an issue to be resolved.

A great example of how something minor can become a major frustration has been happening for a while now. clone, a nice little lib that lets you pass pretty much anything in, and get a cloned version of it out. I use it in many of my modules, but it has an issue:

https://github.com/pvorb/node-clone/pull/46

clone checks if the input is a Buffer, and if it is, it will clone the buffer. This is required functionality of the module. The issue is, even if you never mention Buffer anywhere else in any of your code, or your dependencies, the fact that clone *can* deal with Buffer, means browserify *will* include it.

Programmatically, this  is fine, but the browserify Buffer implementation is reasonably large (40kB!), and adds a significant weight to any browserify bundle that contains it.

I realised that trying to obscure Buffer from browserify wasn’t going to cut it, and decided that a better solution would be to solve this in browserify.

The actual problem

The root of the issue is that browserify should NOT add Buffer on clone’s behalf. Clone does NOT need Buffer unless some OTHER module is using it.

I opened an issue in browserify explaining the issue, and suggesting a flag be added to solve the issue:

https://github.com/substack/node-browserify/issues/1228

The solution proposed involved adding a field to the “browserify” object in a modules package.json, that would tell browserify what builtins this module uses, but should not be bundled because of it.

The discussion finally petered out with basically “That sounds hard”. I had a dig through browserify, looking to add this myself. I was able to figure  out that it was actaully going to be insert-module-globals that needed some work, but it didn’t have access to the package.json, so the option would need to be passed, and the code was so unreadable that I eventually just gave up.

The TL;DR:

some modules, like clone need Buffer to create Buffers, but only if another module passes it in.

  • if no other module in the bundle references Buffer, clone does not need it, so it should be omitted.
  • if another module in the bundle references Buffer, clone may need it, and it will be available, because the other module will have caused it to be added.
  • Browserify needs a way to be told when a module shouldn’t cause a builtin to be bundled, just like peerDependencies.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s