Pushdata.io blog


Posted by Ragnar on Jan 28, 2019

Pushdata.io was developed using vanilla HTML, CSS and Javascript. It does not use any kind of framework and does almost completely without JS components too. There is no jQuery, no Backbone, no Angular. The only external things the site uses are:

  • Google Analytics for statistics
  • Stripe (stripe.js) for payments
  • Highcharts (highcharts.js plus two Highcharts modules, also from highcharts.com) for generating graphs

Or, to put it another way: components are used only when building something from scratch is close to impossible

In total, the code amounts to 1,500 lines of Javascript (without minification), 1,000 lines of CSS and 500 lines of HTML.

Much of this code is copied and pasted, unoptimized and in great need of refactoring. But:

  • It runs
  • It is fairly fast and efficient
  • It looks decent, I think :)
  • It works in most browsers
  • It is responsive
  • It has a unique look and feel
  • It didn't take much time to create

There are other benefits also (and drawbacks, of course) to doing things this way, and I'll get into those later but before going on about that I'd like to take a moment to explain why I chose to go this route.

Why I did it

Why would you not use all the incredible infrastructure out there? All the front-end components and frameworks? The things that allow you to build a login form in 5 minutes, to create a beautiful-looking, cross-browser compatible paginated table in the blink of an eye?

Well, one reason would be that you don't know any of them. And this seems to be a good time to admit that no, I'm not the most experienced front-end developer in the world...

The not-quite-10x developer

While I wrote my first HTML sometime in 1995 or 1996, I haven't exactly been at it every day. In fact, if you add up all the time between 1995 and 2018 that I've spent doing front-end work it has probably been about..PI days. This means that up until last year, my knowledge was firmly stuck in the 90's.

Last year I knew things like how to use the <CENTER> tag for, uh, centering stuff or that you should use BORDER="none" if you didn't want a stupid border surrounding your images. I had heard that <BLINK> was out of fashion (and now I know you have to do it with CSS these days, if you want to resurrect an old and horrible past). Oh yeah, and note how I write tag names all in uppercase? That's also me being influenced by HTML code seen in the wild in the mid-90's.

Luckily (?) for me, the past 20 years I've mostly worked either with back-end development, or in various management positions, so this lack of up-to-date knowledge hasn't been a big problem.

In 2018, however, I quit working for the company I had founded - Load Impact - to be unemployed and build new things. I had realized that building stuff was what I really enjoyed doing, and wanted to get out of management. For maximum possible freedom, I wanted to do as much as possible on my own. The back-end/server side of building something has never been a problem for me. I recently started using Go and have found it to be a very nice language to build back-end services in.

But, as I found out, and to my great surprise, these new things I wanted to build often seem to require a user interface of some sort. It really is a cruel world. At first, I tried hiring freelancers on Upwork to produce basic HTML+CSS and thought I'd just be able to sprinkle a few lines of Javascript on top of it all, to produce a working SaaS app. I didn't have to learn stuff like HTML5 (having missed versions 3 and 4) or CSS (apparently at version 3 already!) just to build stuff, did I?

In theory, perhaps it's a yes, you can "just add Javascript", but not in practise. To be able to glue things together and produce a holy HTML/CSS/JS trinity that works and is usable by people, you have to have basic knowledge in all three areas. I kind of suspected this, so procrastinated a bit in attacking the front-end problem.

The React-Redux experience

Then I got an offer of help from some web dev students who needed a project to work on as part of their training. I thought it sounded like a good way to get started with the scary front-end stuff, having them build a version 1.0 of the front-end and then I would just learn from them how to do it, and take over and finish it. How hard could it be?

The students did a pretty good job, and then delivered a codebase with the most basic functionality of the site/app. The whole thing was built on React+Redux and I sat down and started studying it, trying to figure out how it all worked. It was 4,500 lines of code in 100 different source files. While a lot of files, the whole codebase looked quite nice, modular and consistent. React and Redux too looked to be very consistent, well-documented and shouldn't have been too hard to get into for a longtime developer like myself.

But it was! I spent about three days reading the code, reading tutorials and docs for React and Redux, and still felt quite far from grasping how the whole thing worked, which code did what, where to go to find some specific thing in the code. It wasn't this particular codebase, I realized - it was a combination of several factors that made it all hard to grasp for me: the complexity of React+Redux, the fact that I hadn't used any other front-end frameworks, and the fact that my basic front-end skills in HTML/CSS/JS were lacking. I saw that it would take me quite a while to get productive from the point where I was.

Doing things the "proper" way would mean:

  1. Polishing up my HTML/CSS/JS
  2. Learning React and Redux
  3. Start working on my app
Impatience is one of my virtues, and three steps were at least two too many, so I decided to perform an experiment. What if I instead just:
  1. Started building the app in vanilla HTML/CSS/JS!
Hah! No stupid preparatory steps - I would make progress more or less immediately, as I had some limited HTML/CSS/JS knowledge that would get me started. Then my skills would improve along the way. I might realize after some time that re-creating the functionality of the React+Redux app would be too much work without some kind of framework, but I decided to see how much I could build in a week, if I started from scratch and without any framework.

Vanilla rulez!

After just two days, I had something that looked exactly like the React+Redux app, but without any functionality. Basically, I had just created HTML and CSS to get the right look, but that was fantastic because HTML and CSS were my weakest spots! So that progress inspired me to continue and after two weeks I had replicated the whole React app, using just HTML/CSS/JS!

The codebase was at that point just 1,000 lines of code in 3 files (down from 4,500 LoC and 100 files for the React+Redux version) and also much lighter for the clients to download. The React+Redux app had been almost 6 MB in size when compiled, while the vanilla app was ~400K, which made a huge difference when on a slow connection. When on a 1Mbit/s WiFi connection in Greece, my vanilla app loaded in 4 seconds and started rendering after ~2 seconds, while the React/Redux app took 50 seconds to first render!

The biggest benefit for me at the time, though, was the fact that I now knew the codebase inside and out.

Now, me being a less-than-fantastic front-end developer meant that my codebase wasn't fit for consumption by others. The browsers would barely swallow it. It would've been hard for a new developer to join the team and learn it. To be honest, they would probably have quit as soon as they saw the state of the code. But if a decent front-end dev had written it, I'm sure it could have been made very approachable. In fact, the codebase has improved a lot since the first version. There has been quite a bit of refactoring along the way and while it is still not anywhere near "high-quality" I think it can get there eventually.

Main takeaways

One of the most eye-opening realizations here, I think, is how fast vanilla front-end development was for even a newbie like myself. 2 weeks to reproduce a React web app that took a bunch of students months to create! Yes, I have a lot more development experience than they do, but they definitely had more up to date front-end knowledge than me when they started out.

And for those who go "bah" when they hear it took me >1day just to create a paginated list: Yes, I do know people who are expert users of some framework, who will whip up the basic functionality of a whole web app in a day using that framework.

But: the devil's in the details!

That is, while a framework will let you create standard solutions to standard problems very quickly, you always then have to add that little special sauce that makes your app unique, and that is where the frameworks can slow you down instead of speeding you up. Vanilla coding, on the other hand, is just as fast whether you're doing something "normal" or something unique to your particular app.

I'd argue that sometimes the best solution, for a seasoned front-end developer, may be to build their own library of reusable code that they know inside and out, standardise on vanilla coding and pulling in code primarily from their library when they realize they have something there they can use.

The difference between doing this and using off-the-shelf standard components may not look big, but it is perhaps important:

Using your own code means it is easy for you to change or extend it when you have to. Using a standard component will make you very reluctant to create unique solutions based on modifications to the component - you'll not want to risk your app breaking because of future updates to the standard component. Also, in most cases you probably don't know the component well enough to feel you have the time or ability to modify it, even if you wanted to.

Vanilla coding encourages you to make your app unique, while standard components encourage you to make your app conform, blend in.

Ergo: Stand on the shoulders of giants only when the alternative is to muck around in the dirt! ;)

*I* am definitely going to continue with my vanilla coding. I'll refactor and break out existing functionality here so it gets easier to reuse for other projects, or in new pages on the site. This blog section was the most recent addition to pushdata.io and took me a few days to build, mostly because of the coded-from-scratch blog comment system and quite a bit of refactoring that I also merged back into the other pages. I'm pretty confident no other site anywhere has a blog that looks like this one.

The hardest things

"We choose to do these things, not because they are easy, but because they are hard..."

So, what was hard?

  • The "API explorer" on the front page of pushdata.io is a form you can fill in and then press a button to make a request to the pushdata.io API to either store or retrieve time series data. To be more pedagogical (and because it's cool) there is also a fake terminal window where you can see a CURL command that would perform the same operation. The form updates the terminal window live when a user changes the parameters of the form and then when the "Execute" button is clicked, an AJAX request performs the actual API request. When results from the AJAX request comes in, they're shown in the terminal window, to make it look like it was actually a CURL command that was executed.

    This functionality took a few days to implement, to make it look OK and to make it responsive. I doubt any components or frameworks would have made it substantially easier though, as this is such a special case.

  • The user dashboard is the only "real" page in the pushdata.io app. It shows account information and an overview of the stored time series data, plus a chart containing a graph of the selected time series (this chart is not visible on the screenshot - you have to scroll down to see it). The page has quite a lot of behind-the-scenes front-end logic for changing the way things are displayed depending on user status. For instance, if the user is approaching their quota limits, usage numbers will get a red color and an "Upgrade" link will be added - a CTA (call to action) to make the user upgrade to a premium account.

    The table listing, and the pagination functionality, took around two days to complete. Here, a component would definitely have helped but on the other hand - having done the work once I'm fairly sure I'll be able to break away and reuse most of the code pretty quickly if I have to make another paginated table.

  • On the user dashboard, and on the front page, there are tooltips for many of the UI elements. They display context-sensitive help texts to the user when the mouse pointer hovers over a UI element.

    The tooltips took a while to get right. I'm still not 100% happy with them. They sometimes block things they shouldn't, they could look better than they do, etc. Another thing that made them take longer to implement on the dashboard is the fact that the JS code populates some of them dynamically, depending on e.g. how much quota the user has left.

  • There are several modal dialogs that can appear when the user clicks on buttons on the dashboard and on the front page: the "login modal", "change account email modal", "resend confirm email modal", "upgrade account modal", "cancel premium modal". All these modal dialogs work slightly differently, many can be stacked on top of one another, and together they took several days to implement. This despite the fact that they started out as copy paste versions of the first (login) dialog.

    A component to help creating modal dialogs would absolutely have been useful, but maybe also have forced me to create the modals more similar to eachother.

Earlier in the article, I promised to mention good and bad things with vanilla front-end development, so here is an attempt at a list:

The good stuff about vanilla

Reducing dependencies is always a good thing. It gives you freedom and saves time by making it easier to keep track of dependencies, makes it easier to build, ship and maintain your app. For Pushdata.io there is nothing that has to be built using Node, so I have eliminated not only dependencies needed to debug, run and ship the application; I have eliminated Node as a build dependency also. There is actually no build step at all, which allows a browser to just load the front-end directly from disk after cloning the repo.
  • Specialized UI elements, unique look and feel

Not using standard components means a lot of UI elements will be tailor-made for their particular use-case. For example, instead of using a generic modal dialog element for all modals, I have unique modal dialog elements for each different type of dialog (e.g. the login dialog, the error message dialog, etc) that can be very customised for their use case. Frameworks always try to make things reusable, but if you're developing the first version of an app it is likely not so big and you may only need one instance of a certain component. Your framework will limit to what extent you can customise this component, while vanilla coding will not. Specialized elements, in the hands of a skilled developer (not saying that developer is me!) will be superior to standardized ones.

More or less the same as the above, but more generally stated: Avoiding standard components that limit your degrees of freedom when it comes to designing the user experience means your app will be unique. If you're competent, it can result in a better and more selling app. If not, well, you should probably use a framework! :)

  • More compact, higher-performing app
Again, these benefits probably depend on the skill of the developer. If you know what you're doing you can always outperform a generic framework on most metrics that matter by hand-coding things. In the Pushdata.io case it lets me have probably more fine-grained control over AJAX calls and data refresh than I would have with some frameworks that do these things in a "smart" way, it probably means I don't have to include a bunch of unused framework code, bloating the download weight for the client, and stuff like that. Given my inexperience in front-end coding it also probably means there are some functions I currently perform less efficiently than a framework would.
  • Full control/full knowledge of codebase

Things that are hard to quantify are often overlooked, and this one probably belongs in that category. Being in full control over what your code does - knowing exactly what happens when and where - is something that can greatly help in tricky debugging situations. An advanced framework like e.g. React will perform a lot of magic for you that you don't understand, which means you're likely to be quite lost when something goes wrong. Thank god for Stackoverflow, eh?

  • More future-proof skill learning
Who knows if React will be used 5 years from now? Is the time you invest in learning your framework and your components well-invested time, or will you have to relearn a new framework 2 years from now? Vanilla HTML/CSS/JS is basic web developer knowledge that doesn't age as quickly. Even I, who started out with a lot of 20-year old knowledge, was able to use it to produce something.

The bad stuff about vanilla

Depending on how well you organize your codebase, how good your coding standards are and how consistently you follow your own rules, you may experience issue onboarding new developers into your project. Nothing new under the sun though - this is the reality in most software projects. It's just that a framework can help you adopt good habits if you're lazy, ignorant or just have a dev team that is >1 person :) This one used to be a big argument for using frameworks, because the framework would guarantee that when you used feature X it produced the same result on all supported browsers. However, people I know (who're supposed to know these things) tell me browsers have become much more standardized and in general better-behaved, which means that cross-browser issues aren't nearly as common or as bad as they used to be. My experience is the same - I haven't had many cross-browser issues at all while developing Pushdata.io in vanilla HTML/CSS/JS. Getting the app to look OK on devices with widely different viewport dimensions - i.e. working on responsiveness - is something I expected would make me run into lots of cross-browser issues but I can't say I spent much time on this at all. Max 5% of my time was spent solving cross-browser problems.
  • A large app may benefit from more modularization

A framework is built to be generic and modular. It is likely that a completely home-built codebase will not be as modular or well-suited to creating tons of similar UI components from the same basic component template. For large apps a framework can be very useful. For small MVPs, prototypes and startups wanting to iterate fast on a small codebase, vanilla coding may sometimes be better.

This is where some people will refuse to even consider the vanilla option, because they foresee the problems they'll have later on, when the codebase grows, the team grows and app usage grows. I have just one counter-argument, but it's a good one: premature optimization is the root of all evil