For me, the illuminating item was this:

main thread availability required for painting updated image data

imgur has an <img src="…"> right in the HTML. This allows the network request to kick off right at the start. (That's awesome and a huge improvement over the m.imgur of two years ago).

Images resources are big and can easily be over 50k, thus taking some time to finish the download.

But it turns out that the main thread must be available in order to paint any new data from an image. And since the main thread is completely swamped with a big block of javascript, it can't update the screen. I saw one example where the image download finished just 5ms after a 3000ms long script block started.. Had the network request finished just a tad earlier, the page would have visually loaded a whole 3 seconds earlier.

Appreciate the audit - we're still iterating and expanding the rollout and addressing performance as we can (tuning post-load performance for slower devices and server rendering behavior have admittedly been higher on the priority list so far). #3 seems like a farther off possibility, but 1 and 2 seem pretty reasonable.

PS: Tony says hi.

Hi! Thanks Sam for this audit, it is always great to see how we can improve.

Now I don't know about you but 2 seconds between me and Cage is simply unacceptable.

Sounds like a reasonable motivation 👍

(Anyway, thanks for the insights)

Use a tool like rollup or Nolan Lawson's new rollupify https://github.com/nolanlawson/rollupify/ to drop the overhead (40% of the JS time) of browserify

@samccone (or anyone else): what's the current best practice in this case for webpack users? Would running Clojure Compiler on the code generated by webpack yield any improvements? I recently did a small experiment with this which did make the file smaller but I didn't evaluate whether this module loading logic would be simplified.

Use a tool like rollup or Nolan Lawson's new rollupify

I don't think that rollup will speed up the execution dramatically. It reduces the ungzipped file size, yes, but all the code still needs to be executed. There are just less function calls for the module initialization but I'd bet that this is not the main part.

what's the current best practice in this case for webpack users?

I think, it's best to defer module execution as much as possible. So, inside your entry files you should not just require() or import() all modules immediately. It's better to use asynchronous module loading, like require.ensure() or System.import() as soon as the modules are actually required (or inside requestIdleCallback).

However, I don't think that this is specific to webpack. It applies to all frontend JavaScript developers that have to deal with a large codebase.

Another possibility is to move as much stuff as possible into a separate web- or service-worker.

@jhnns From the screenshots it looks like Browserify's module loader is taking up a non-trivial amount of self time (~10-13%), so a tool like Rollup should indeed be able to eliminate that (since there's no loader; all deps are just initialized in the same scope).

That said, I think you're right that @samccone's 40% figure is exaggerated, since it seems based on the total time instead of the self time. And I agree with all your other points.

There are just less function calls for the module initialization but I'd bet that this is not the main part.

^ in fact the function call overhead is significant and by using a tool like rollup it should significantly drop the module wrapping overhead cost. The only reason I feel confident saying yhis is because by looking at the timeline and profile we can slice up exactly where the time was being spent.

(refer to https://docs.google.com/document/d/1E2w0UQ4RhId5cMYsDcdcNwsgL0gP_S6SDv27yi1mCEY/edit for a more detailed look at the module overhead)

As far as self time vs total time in this example, I used total because when I walked the down the "boot" tree no one was doing any work, it was simply the passing of the require vars into new closures, again I would refer to the above doc for a closer look but this module boot time overhead seems to be endemic to a majority of web apps out there.

Refer to @cramforce 's win that they got by switching to a tool that did topological sorting of dependencies:
https://twitter.com/cramforce/status/690640077865619456

If you would like to compare the raw output btw these tools checkout
https://github.com/samccone/The-cost-of-transpiling-es2015-in-2016

tl;dr The module overhead is significant

@samccone thanks for the additional information. Really interesting! 👍

I know that browserify can be optimized by removing the path resolution stuff.

Does anyone have more experience with rollup? Is it "valid" to put all stuff into the same scope or does it introduce additional problems? Personally, I would have a bad feeling to change the production code this way, but it might be a valid approach, just as minifying it. And what are the improvements in execution time in real-world examples? Is it worth it?

There is ability for WebPack / Babel to perform optimizations, such as dead code elimination (aka Tree Shaking) or dropping strict compliance, though currently the setup is complex, and documentation gets scattered.

I know that browserify can be optimized by removing the path resolution stuff.

Is there a plugin or something? Or are you just talking about bundle-collapser, which it appears Imgur is already using?

Does anyone have more experience with rollup? Is it valid...

ES6 module semantics mean it's perfectly valid to hoist everything into a single scope, and yes it can have big savings. I've written about this here and here. I've also gotten some good feedback about rollupify here. I've also used it on a small internal project and saw a more modest gain (<5% range).

Or are you just talking about bundle-collapser, which it appears Imgur is already using?

Yes, I'm talking about bundle-collapser. It makes sense to turn paths into IDs because this can be done at compile time.

@nolanlawson thanks for linking these interesting articles. I mostly agree with all your points 👍

Closure Compiler can also do scope collapsing for CommonJS modules. There may be some edge cases, but in general this works really well. Configuration is pretty tough, though.

Here is a sample PR that turned this on for my project AMP: https://github.com/ampproject/amphtml/pull/1218/files
This is switching from Babel/Browserify to Closure Compiler for production codegen. We continue to use Browserify for development and have no plans in changing that.

Just read this, wondering if these ideas could have any impact:

1. Give up the main thread at certain points in the code (for example: where browserify loads a module). Could do this with setTimeout. Then the browser would have more opportunities to write images etc to the page.
2. Initialize the JS in an onload on the image itself. Most important content gets loaded and displayed to the user, then you init JS and enable interaction with the site.

@derekkraan your second idea is exactly what @samccone suggested as a fix (his #1 suggested solution). Using setTimeout to halt the JS execution can be done, but you would have to know exactly when and for how long to call it, and on an mobile connection that would be tricky, as it's generally of a lower quality and more prone to sudden drops.

https://twitter.com/csswizardry/status/1185604806901207045?s=19

a simple solution would be to load the img first and then load the ads/comments and all the bloat stuff.