Agile CSS and Frontend Testing at the JavaScript Days

We attended the JavaScript Days (JSDay) in Verona and I summarized my favorite talks for you to enjoy. After the opening chat by the GrUSP guys – did you know they not only organize PHPday and JSday, but a number of other smaller conferences like better software and kerning? – the madness started:

Table of Contents

• Metaprogramming in JavaScript
• NSA.js – really tracking user interaction
• Flow-Based Programming for JavaScript
• The spirit of testing
• Frameworkless JavaScript with NPM and browserify
• How to *actually* use promises in JavaScript
• Understanding scope in JavaScript
• Agile CSS development with Compass/SASS
• Frontend acceptance testing frameworks
• The Internet of things, powered by MQTT and Node.js
• Generative art in JavaScript
• Automated CSS-testing with JavaScript (really)
• Performance testing crash course

Metaprogramming in JavaScript

Massimiliano Mantione, who previously worked on v8 team at Google, starts the day off on a skeptic note: “We all know that JavaScript is powerful but it can be a really terrible language when misused. It lacks several advanced features that are taken for granted in other languages. One technique that is overlooked is metaprogramming.” Mantione developed Metascript taking the best bits of Coffeescript, Typescript, Lispyscript, Clojurescript, Lisp and Haskell. Code that can modify your code, that compiles to JavaScript…

Metascript has a readable syntax and at the same time allows for lisp-style metaprogramming (macros that can manipulate the AST). Lisp-like languages have a ‘clean’ syntax per definition: they explicitly use parenthesis everywhere to express the grouping of expressions. Metascript uses a combination of parenthesis, indentation and infix operators to ensure that the syntax is similar to a number of mainstream programming languages, and can be used interchangeably. The next steps for Metascript would entail completing the compiler and the metaprogramming system. Writing standard macros and publishing a module on GitHub are also on the road map. As is implementing a type system. One to watch.

NSA.js – really tracking user interaction

Danni Friedland (eBay IIC) dislikes how current tracking solutions are sub-optimal. Charts, graphs and heatmaps are too abstract. There’s no up-close-and-personal with your user. What if you could see what the user sees? You’ll have to record everything and add an Eventlistener to scrollObserver, mouseObserver, ViewportObserver (screen sizes), TestSelectionObserver, track focus in/out, URL changes, the open or closing of a popup…

To see how the page changes, on a DOM level, say hello to Mutation Observer. Praising its great browser support, and how it’s asynchronous, aggregated and fast, Danni went on to explain how Mutation Observer gives us child/root observation, attribute changes, content changes, old and new data. “A single mutation record gives us added/removed notes, type of change, target of operation, previous and next sibling nodes, the value before the change and the name of the changed attribute. Zero interaction with your code.”

Yet it is slightly limited. You can’t see CSS changes (like :hover), the shadow DOM conflicts with the observer and you won’t see anything you could not view in the inspector. Plus: there’s no timestamp. Danni solved this by serializing nodes adding a unique id (running number). Storing the initial view size, scroll position, mouse location, DOM tree and user selection, he can then start listening for changes (and save each event to the server). When replaying, Danni uses an iframe for the viewport size and replays at the appropriate timestamp. His demo is really quite extraordinary but he admits there are some limitations. “Assets (images, fonts) are hard to serialize and we can’t view changes made to external CSS.”

Flow-Based Programming for JavaScript

Henri Bergius is the author of Create.js and NoFlo. Decoupling software one piece at a time, he created NoFlo as a flow-based programming environment for JavaScript. With NoFlo developers build their applications wiring data streams together between different pre-built or custom components.

Henri’s aversion towards ‘callback Christmas trees’ made him follow Paul Morrison‘s example of designing flowbased programming. He also looks to Alan Kay (OOP pioneer), who thinks of objects as biological cells. “And I like the Unix philosophy of connecting smaller programs. If you ever used the Unix shell you did FBP in 1D.” His Flowhub (the result of the 2013 Kickstarter campaign) released last week and is quite impressive: flowhub.io.

The spirit of testing

Marco Cedaro, working in the webteam at Shazam, describes Front-end JavaScript testing as ‘the ghost of the developer community’. “It has been a thing since 3/4 years now and still some people don’t believe it exists.” Marco claims that you are testing in your head all the time, structuring your code with if statements. Therefor it’s not a discussion about tools or frameworks, these are mere implementations. “You need a strategy. And Continuous Integration. But for that you’ll need to identify what to automate.”

Answering obligatory questions like how much code coverage you have, or how long it takes for your tests to run, Marco says ‘any test is better than no test’. “The sooner you get that feedback loop the better, to have you not working on old code.” Unless you write your tests after writing code rather than upfront (DTU), only to get a cheap confirmation of your code. “You write unit test because you want your code to perform and you are doing that for yourself. No client will ever ask you to do testing.” Marco shared how he discovered that there’s more than just unit and integration testing. And how not everything can be automatized (just think about code reviews). To illustrate that automation is not always the answer, Marco played the moonwalking bear video.

Frameworkless JavaScript with NPM and browserify

Rob Ashton (self-proclaimed ‘Erlang and enterprisy JS guy’) talked about the common problems with frameworks. Like the where-did-this-function-come-from issue (spoiler: from framework x), the order of including files which apparently matters and finding your entry points. NPM is written for server-side code but Rob wondered if he could use it for client-side as well? “It might be synchronous and thus not necessarily compatible for the web, it is going to involve a build step and it might make debugging a bit harder, but we will just take the hit.”

Demo-ing Browserify, which packages our JavaScript for us, Rob explains how we can include templates via transforms and ‘shrink-wrap’ modules from NPM. He warns that it is a hack, but a good one. “Frameworkless doesn’t mean not using other people’s code nor re-inventing the wheel. Frameworkless is about enabling flexibility and regaining control.” Your new conventions, with the compliments of Ashton:

1. 1. Use a single entry point; ‘app.js’
2. 2. Use a routing library to kick off individual pages
3. 3. Utilize feature folders, not pattern folders
4. 4. Bundle your assets from your feature folders
5. 5. Avoid relative paths for dependencies
6. 6. Treat everything as a self contained module
7. 7. Avoid deep dependencies
8. 8. Avoid large dependencies
9. 9. Take ownership of your dependencies
10. 10. React makes a fine view engine

How to *actually* use promises in JavaScript

Jakob Mattsson talks promises. “They have been around for a long time and even made it into ES6.” The Node.js WebDriver before promises looks like crazy talk, but with promises it’s spammed with ‘then’. Promises should make async code as easy as sync code and Mattsson claims that it is indeed possible, but ‘they don’t tell you that in the tutorials’.

How? Always return promises, no callbacks. Functions should always accept promises as well as regular values. And do augment promises as you would augment objects. Mattsson created the z utility library, which resolves any kind of object/array/promise/whatever, makes functions promise-friendly and features jQuery (or underscore) like extensions.

Understanding scope in JavaScript

David Aragon (QuickLeft) understands scope. “Scope is one of the most critical concepts in JavaScript, but even seasoned front-end developers have trouble with the the meaning of this.” Aragon explains scope as the lifetime of a variable. “JavaScript does not use block scope like C and Scala, but functional scope. For a variable to be local it needs to be declared with var and it must be declared inside a function.” A variable is global if it is not declared with var or declared inside a function.

Aragon on why exactly global scope is so bad: “Everyone has access to the global scope and 3rd party code can change your values at any time.” And if that does not convince you, he recommends you go ahead and read this. If a function is a sentence, the context is the subject of that sentence. It’s who the function is about. We access the context through the variable this. We’re used to lexical scoping (where we can see the value of a variable in the visual scope). this uses dynamic scoping, which we’re less comfortable with.

Agile CSS development with Compass/SASS

Andrea Verlicchi complains how CSS can get messy quickly, with many site and page sections, plus the multitude of  devices, screen resolutions, colors, fonts and helpers. Without an unique author or coding style, CSS files become long, verbose and unreadable. CSS is not a complete language, it misses variables, extensibility, very few functions, nesting, partials and math. In come SASS and Compass.

SASS or SCSS (the latter being fully compatible with CSS) support nesting (of rules), allows to create variables for colors (like $mainColor; #FFF), has functions, partials (@import "whatever";), mixins AND math (percentages of margins and widths for instance).

But SASS also produces CSS you didn’t actually write. Compass protects you from unpleasant surprises. What is more is that it is loaded with helpers and patterns (sticky footer, CSS3). Verlicchi recommends taking the 30 minutes it will take you to learn Compass. Today, if possible.

Frontend acceptance testing frameworks

Everyone knows that acceptance testing is important, but few people enjoy writing acceptance tests. Vikki Read knows this all too well. Thankfully, several frontend acceptance testing frameworks have emerged in the past year, all designed to help you write clear, expressive, JavaScript-based tests that make sure your code works everywhere it runs.

Arguments against testing Read frequently heard are ‘too much overhead / slow’ (‘then you are probably testing the wrong thing or you structured your code wrong’) or: ‘but browser x is hard’. Praising testing frameworks dalek, nightwatch and intern for their expressive chainable interface and BDD-style test writing, Vikki warns that they use promises under the hood. For a full-on frontend acceptance test frameworks comparison, check out her README. Vikki also created dalekmob, nightmob, internmob to her requirements and they are well worth checking out.

The Internet of things, powered by MQTT and Node.js

Matteo Collina likes his hardware. MQTT is a publish/subscribe protocol for the Internet of Things (IoT), which roughly means that using MQTT, you are able to send and receive messages to and from things, directly in your JavaScript application, both in Node.js and in the browser. You can even receive the messages from your ‘thing’ if you are offline.

Matteo praises the instant gratification that comes with playing around with MQTT (in combination with a SensorTag for instance) and encourages everyone to start playing around with code. He is a big fan of Mosca, a MQTT broker in Node.js, and recommends using Ponte in case you need a REST API.

Generative art in JavaScript

Enrico “Catodo” Zimuel calls all attendees to ‘have fun with JavaScript’. Starting with a live performance of his ‘random cuts’ project, he got everyone excited about creative coding. Catodo explained how generative art refers to art that ‘in whole or in part has been created with the use of an autonomous system’. His examples use (simple) algorithms, Leap Motion and other hardware and the power of JavaScript, HTML5 and CSS3. Libraries such as processing.js, D3.js, Paper.js, three.js and Raphaël also make for some great tools creating generative art.

Automated CSS-testing with JavaScript (really)

“We all love building on top of declarative programming languages like HTML and CSS. We created SASS, Less and Stylus to add more power them. I believe we focus too much on writing versus reading code.” CSS files are often a mess,  Jakob Mattsson rightly claims. “New styles are added instead of reusing existing code, unused styles are not removed. We often have no overview of how different styles interact.” Jakob differentiates between two types measures for code quality; functional quality (its fitness for your purpose) and structural quality (reliability and maintanability). To ensure quality we write automated tests that follow a certain timeline: write a test that fails – make the code work – eliminate redundancy (refactor). Yet we don’t bother to do this for our stylesheets.

Before you say that there are no tools to do that, you are wrong and you should feel bad. Jakob lists 4 approaches:
1. First off, you can check your syntax to see if you actually wrote valid CSS. Your editor, IDE and/or a dedicated plugin come in handy here.
2. Defining style guides and linting, you can (en)force best practices syntactically and semantically. It prevents anti-patterns and it’s easy to apply, yet you are not testing the end result.
3. Screenshot tools like Huxley compare screenshots and detect change. There is no way changes can sneak by and it’s an implementation agnostic solution. Yet you’ll have to prepare for a lot of false positives.
4. Or you can snif the DOM for all relevant styles and benchmark them to your expectation (for instance your .psd file) with a tool like Hardy. In Hardy you define scenarios much like Rubyists are used to doing with Cucumber. It’s resilient to change and tests ‘the live thing’. But it is tough to write expectations in code when style rules are hardly ever interpreted the same way in all browsers.

Jakob recommends using a combination of all the above. Or: building a solution that does that for you (and the rest of the frontend world).

Performance testing crash course

Dustin Whittle worked at Yahoo and Sensiolabs previously and is now employed by Appdynamics as a developer evangelist. “Microsoft found that Bing searches that were 2 seconds slower resulted in a 4.3% drop in revenue per user. The performance of your application affects your business more than you might think. Unfortunately, most engineering teams do not regularly test the performance and scalability of their infrastructure.”

So how fast is fast enough? 57% of online customers will abandon a slow page after 3 seconds. 0.1 second feels instantaneous, up to 1 second allows users to think seamlessly, but it really goes downhill from there. Dustin Whittle shared the latest performance testing tools; Siege, MultiMechanize, Bees with Machine Guns, Google PageSpeed, and many more.

With You have ruined JS Rob Ashton took on the closing keynote. “It’s time to have a conversation about JavaScript.” Rob doesn’t like to fanboy over the angulars and embers and backbones and model view controllers of this world. From the ‘coincidence’ that Take That and JavaScript gained traction around the same time, via polished turds (read: ember.js applications) to json voodoo, Rob spares nothing and no-one. Even Google has to hear it, as ‘the root of all problems’. Ashton calls for reflection and after a conference with ‘awesome’ being used at least 3 times every talk, JSDay could not have wished for a better ending.