The HUGE Grunt.js Guide to a Better Frontend Workflow (Part 1: Coding)
Preface: This is the kind of guide we wish we had when we were starting to learn Grunt. It's lengthy, so if you want to skip that, there's a handy gist that goes along with it so you can share in our process.
This guide is separated into 3 parts. This is Part 1 and covers our local development process. Part 2 covers our testing process, and conclude the series with how we deploy with grunt.
Let's face it, development work at times can be repetitive, mundane and just flat out not fun. A lot of that comes from the tedious tasks that can come from optimizing front end development: minifying, concatenating, copying, refreshing, optimizing, uglifying, and gasp testing! But fret not, there's hope! Tools are available that limit (as much as possible) the annoying tasks of development. Grunt: The JavaScript Task Runner is one of those tools. There are many like it, but Grunt is our tool of choice with its large community of support. While we'll be focusing on Grunt, many of these tasks are available in those other build tools like Gulp. If you're not using Grunt, you might find some inspiration!
At SRC:CLR, our platform is built with a Single Page Application (SPA) architecture. Our front end code is entirely separate from our back end. One of the benefits we're seeing on the front-end is in our build process. All those tasks above? Yep, those are in our Grunt toolbox. We're even deploying from it (for now).
This guide is organized a little differently than most. It's transformed into Grunt: The Definitive Guide of the build tasks we use at SRC:CLR. We'll walk through what they do, explore new things, and gotchas along the way.
The Three Commandments
Ok, they're not really commandments... But, we use Grunt to run three primary commands.
- grunt serve - local development
- grunt test - making sure we're not doing something awful
grunt deploy- optimizing and deploying
Our development process centrally focuses on these commands and we've separated this guide along those lines. Here's to better workflows!
grunt serve
This is the base command we use for running our environment locally. With it, we're able to load our site, connect to an API server, automatically refresh our browser when files change, and even conveniently verify if our build process works.
The base set of commands are broken down like this:
grunt serve- runs everything locally and assumes you have a local backend environment runninggrunt serve --api=qa- same as the above, but allows you to specify a preconfigured API servergrunt serve --apiHost=www.google.com --apiPort=1337- if you haven't preconfigured an API servergrunt serve:dist --api=qa- verify your build process locally off a near production environment
These commands are only intended for local development. The base task is registered in our Gruntfile.js:
grunt.registerTask('serve', 'Compile then start a connect web server', function (target) {
if (target === 'dist') {
return grunt.task.run(['build', 'connect:dist:keepalive']);
}
grunt.task.run([
'clean:server',
'injector',
'configureProxies:livereload',
'wiredep',
'concurrent:server',
'ngtemplates:serve',
'autoprefixer',
'connect:livereload',
'watch'
]);
});
A lot going on in this little snippet, so let's break it down. (If you're asking about the api parameter, we'll cover it below.
grunt.registerTask
While we use registerTask in an unusual way, our approach is still straightforward. Usually you'll see something like grunt.registerTask('default', ['jshint', 'qunit', 'concat', 'uglify']);. In our instance, we're using the function syntax. This syntax accepts arguments on the grunt command allowing you to specify a target when you run a task, e.g. grunt serve:target. In the above case, we're using it to serve a default local environment with grunt serve (not optimized) or our build environment with grunt serve:dist (optimized). The difference provides us a more realistic representation of how our code will look and feel on in our production environment.
Let's go over what happens when we just run grunt serve. You can find grunt serve:dist in the part where we talk about deployments.
clean:server
grunt-contrib-clean basically empties a directory. We're simply using it to clean out our .tmp directory so nothing old is being served. The configuration is extremely simple:
clean: {
server: '.tmp'
}
injector
Injector is new for us. We were originally maintaining all of our JavaScript script tags manually in our index.html and, and as you might expect, that became pretty awful quickly. So we automated it. Our configuration is fairly simple but had a few gotchas:
injector: {
options: {
ignorePath: '<%= yeoman.app %>/'
},
dependencies: {
files: {
'<%= yeoman.app %>/index.html': [
'<%= yeoman.app %>/**/*.js',
'!<%= yeoman.app %>/**/*.test.js', // no tests in client... duh
'!<%= yeoman.app %>/scripts/segment.js' // any files in our <head> area
],
}
}
}
And in our index.html where you want your scripts injected:
<!-- injector:js -->
<!-- endbuild -->
Sidenote: What's that new <%= yeoman.app %>? You can use Grunt's template processing to use variables in your configuration. In this case, yeoman is a property set in our initConfig Object.
injector directory gotcha
By default, injector will make our src attribute look like: "/app/scripts/segment.js". This is a problem for us, as we run our local environment with /app as the root directory. So we've configured the above to ignorePath: '<%= yeoman.app %>/', which will now strip it from our src attribute.
injector dependencies gotchas
When we first ran the injector we realized our test files were being included directly into our page! That's definitely not what we wanted.
Not only that, we also realized our code calling back to Segment.com for our analytics was being included multiple times. This was due to it being included explicitly in our <head> element and then injected by our injector again.
Luckily, Grunt allows you to negate matches on files. So we added these two cases to be negated and everything was fixed!
wiredep
We use Bower to manage some third-party dependencies (Angular, Lodash, modernizr, etc.). Just like we don't want to manually include our own script files, maintaining Bower components is cumbersome too. So we automated it with grunt-wiredep, which injects Bower JS/CSS into our page. Here's the code:
wiredep: {
app: {
src: ['<%= yeoman.app %>/index.html'],
exclude: ['bower_components/bootstrap-sass-official', 'sourceclear-style-guide'],
ignorePath: /\.\./
}
}
And, again in our index.html:
...
<head>
...
<!-- bower:css -->
<!-- endbower -->
...
</head>
<body>
...
<!-- bower:js -->
<!-- endbower -->
...
</body>
wiredep ignorePath
We use absolute paths in our projects, however, wiredep by default wires up the relative path to the bower_components folder. In our case, the bower_components folder is served from the root directory, so we needed to remove the ".." that was being prepended onto the relative paths. Luckily, wiredep configuration options which allow you to remove this part of the path. To do this, we use a regular expression to match for the two periods in the relative paths, which are then stripped by the plugin to generate the absolute path we want.
Sidenote: It's been a while since configuring this one. Code comments would've helped me understand what the ignorePath is doing much faster than the time it took me...
wiredep exclusions
Sometimes you don't want to inject certain things. For us we don't want to inject the bootstrap-sass-official component, nor our own style guide as we have manual processes for including these.
concurrent:server
Our first grunt serve-specific task! To be honest, it's nothing special and a place where we could probably improve our performance. For now though, it looks like this:
concurrent: {
server: [
'libsass:server'
]
}
See? Nothing special. Ideally we'd toss more tasks into here and have them all run concurrently, but we haven't made time to optimize this. In the end, all this task does is run libsass:server for us.
sass:server
We use libsass -- C++ compiled Sass! -- for compiling our SCSS. We initially used grunt-contrib-compass but found the processing times to be horrendous for our particular environment.
The task below is simply configured to generate one file. precision gives us 10 decimals when it's doing math. update: true tells it during our watch events to only compile the files that changed.
sass: {
options: {
precision: 10,
update: true
},
server: {
options: {
debugInfo: true
},
files: {
'.tmp/styles/main.css': '<%= yeoman.app %>/styles/main.scss'
}
}
}
ngtemplates:serve
Now this is a fun one that we recently implemented. Since we're creating a SPA, all of our template files were served separately as their own .html files. This is great in theory: it provides the files as they're needed as well as the ability for browsers to cache the templates. However, when it came to new deployments, this convenient caching turned into a nightmare when the cache wouldn't know to check for the new version.
ngtemplates: {
options: {
module: 'SourceClear',
return '/' + templateString;
}
},
server: {
cwd: '<%= yeoman.app %>/',
src: [
'**/*.html',
'!index.html',
'!404.html'
],
dest: '.tmp/scripts/templates.js',
}
}
Sidenote: While we could have opted for filerev our template files, we instead chose to concatenate and compile them into one templates.js file. This achieves a similar effect with minimal filename replacements.
ngtemplates:serve module gotcha
ngtemplates by default attaches the templates via the run method on an angular module. We didn't realize this and were trying to debug for a while. Make sure your module matches your Angular module. If you're intrepid enough, there's an option to have ngtemplates create its own module.
ngtemplates:serve url gotcha
As mentioned earlier, we use absolute paths in our projects, so we set a cwd specifying where these templates live and prepend our templateString (basically the path to the file) with /.
ngtemplates:serve non-template gotcha
We're using that same exclusion syntax from earlier to negate any files that we know aren't templates, in particular our index and 404 page.
An option we're also considering to alleviate this is using an alternative file suffix, like .tpl.html.
autoprefixer
It's 2015. There's just (almost) no need to be prefixing CSS properties anymore. For that, we automate it with grunt-autoprefixer which, in addition to automating this, will continually keep our prefixes up-to-date with caniuse.com data.
autoprefixer: {
options: {
browsers: ['last 3 versions', 'ie 8', 'ie 9']
},
dist: {
files: [{
expand: true,
cwd: '.tmp/styles/',
src: '{,*/}*.css',
dest: '.tmp/styles/'
}]
}
}
Our configuration here is fairly straightforward. We post-process our pre-processed SCSS file and have autoprefixer do its thing. We go back to ie 8 because, why not? It's basically free and if we can make it work for additional users, then that's great.
connect:livereload
Though we've defined the task as having a livereload target, the name betrays it. Connect is a web server that we're starting with the grunt-contrib-connect plugin. The target name, livereload, comes from our intention of using it while working locally. On its own, this task only kicks off our local web server and prepares the browser for livereloading. Our watch task below is responsible for reloading the page. Let's look at how we configured it.
connect: {
options: {
port: 9000,
// Change this to '0.0.0.0' to access the server from outside.
hostname: 'localhost',
livereload: 35729
},
livereload: {
options: {
open: true,
middleware: function (connect) {
return [
require('grunt-connect-proxy/lib/utils').proxyRequest,
modRewrite(['!/api|\\.jpg|\\.gif|\\.png|\\.svg|\\.woff2|\\.eot|\\.html|\\.js|\\.css|\\.woff|\\.ttf|\\.swf$ /index.html [L]']),
connect.static('.tmp'),
connect().use(
'/bower_components',
connect.static('./bower_components')
),
connect.static(appConfig.app)
];
}
},
proxies: {
context: '/api',
host: apiHost,
port: apiPort
}
}
}
There's a lot going on here. Let's break it down:
connect options
Basically we're setting our server to load on http://localhost:9000/. In addition to that, we're setting the port for our livereload instance. On its own this doesn't help us, but in the next task it will!
connect:livereload options.open
It's pretty common to open a browser tab when running a Connect server. Set this option to true and you'll magically see your browser load your development environment. Amazing.
connect:livereload options.middleware
And here we get to some fancy stuff. Our function returns an array of middlewares to run.
connect:livereload options.middleware grunt-connect-proxy
We use this to proxy to our staging backend API server (hidden behind a VPN, of course). While we maintain the option for frontend devs to run our backend environment locally, it's not necessary all the time. This makes it extremely convenient for us to reboot and get back to work.
proxies.context specifies which paths we want to proxy. Our production environment proxies /api to our backend server, so locally that's how we'll want to handle it as well.
The host and port are defined dynamically based on the script we run. We think it's pretty neat:
/**
* Allow the ability to switch between local API and QA API.
*/
var api = grunt.option('api'),
apiHost = grunt.option('apiHost'),
apiPort = grunt.option('apiPort');
if (typeof apiHost !== 'string') {
apiHost = 'localhost';
}
if (typeof apiPort !== 'number') {
apiPort = 8080;
}
if(typeof api === 'boolean') {
console.warn('No API set. Using default localhost option.');
}
if (api === 'qa') {
apiHost = 'qa.sourceclear.com';
apiPort = 80;
}
The grunt.option('api') is defined when you execute a grunt task, such that grunt serve --api=qa will set grunt.option('api') === 'qa'. We predefine our localhost as the default and let developers use our staging server if they want more convenience.
If you were paying attention, you'll notice we skipped over our configureProxies:livereload task. That task isn't defined in our Gruntfile, but is something that our grunt-connect-proxy automatically generates for us. The livereload target corresponds to the connect target we're using.
Sidenote: You'll also notice we ran configureProxies:livereload before starting our connect:livereload task. This is a must to get your grunt proxy to work.
connect:livereload options.middleware modRewrite
You might've noticed that we're calling this modRewrite function. This isn't a generic function included with Grunt. In actuality, we load it at the top of our file because we're using it in multiple places.
var modRewrite = require('connect-modrewrite');
This function is essential to reproducing the behavior we want in a single page application. With it, we're basically redirecting all non-asset URLs to our index.html page to be routed via Angular's routing.
connect:livereload options.middleware connect
For us we have two directories we want to find files from on our root path: .tmp and app. This is simple with the connect.static('path/to/folder') syntax. However, our bower_components folder is just slightly more complicated.
We don't want it to load on our root path, we want it available when someone loads /bower_components. Luckily, Connect can still do that with the connect.static method. As shown above, we specify the path and pass in the folder that contains the files.
watch
And for our finale... clicking the refresh button or mashing F5 is tiring. So we use grunt-contrib-watch to solve that problem for us!
watch: {
bower: {
files: ['bower.json'],
tasks: ['wiredep']
},
js: {
files: ['<%= yeoman.app %>/scripts/**/*.js', '!<%= yeoman.app %>/scripts/**/*.test.js'],
tasks: ['newer:jshint:all'],
options: {
livereload: '<%= connect.options.livereload %>'
}
},
jsTest: {
files: ['<%= yeoman.app %>/scripts/**/*.test.js', 'test/e2e/**/*.js', 'test/karma.conf.js', 'test/protractor.conf.js'],
tasks: ['newer:jshint:test']
},
html: {
files: ['<%= yeoman.app %>/**/*.html'],
tasks: ['ngtemplates:serve'],
options: {
livereload: '<%= connect.options.livereload %>'
}
},
sass: {
files: ['<%= yeoman.app %>/**/*.{scss,sass}'],
tasks: ['sass:server', 'autoprefixer']
},
gruntfile: {
files: ['Gruntfile.js']
},
livereload: {
options: {
livereload: '<%= connect.options.livereload %>'
},
files: [
'<%= yeoman.app %>/**/*.html',
'.tmp/styles/{,*/}*.css',
'<%= yeoman.app %>/images/**/*.{png,jpg,jpeg,gif,webp,svg}'
]
}
}
Now that is a lot! Let's go over it. It's not too tough.
watch bower
Whenever we install/uninstall a dependency, we trigger wiredep to inject or remove the necessary components.
watch js
Whenever a JS file is updated, we run jshint on the files that changed. We use grunt-newer to only run jshint on the files that have changed. Why run jshint only on the changed files? Its not a necessity, but it definitely helps to keep everything running nice and speedy.
We also trigger a live reload in the browser after our tasks are complete.
Sidenote: That <%= connect.options.livereload %>? That's the port we set up above in our connect server. Locally, a livereload.js script is injected automatically to trigger these reloads.
watch jsTest
We treat our test files much like we do our feature files though this time we don't specify a need to refresh the browser
watch html
As you learned above, we recently introduced ngtemplates into our code. We used to just trigger a refresh but now we're processing our ngtemplates:serve task.
We've also considered an html-linter like grunt-html-angular-validate but haven't introduced it yet. Have you or do you use one? Let us know!
watch sass
We run our sass preprocessor and then our autoprefixer postprocessor.
watch gruntfile
This is "supposed" to reload the Gruntfile config as needed. In practice, I'm not 100% sure it does anything useful as I often end up restarting the task anyway.
An optimization point we could add is a jshint task to run and verify our changes aren't awful.
watch livereload
And finally, you might've missed some livereload opportunities above (like our CSS changes)! Not to worry, this task does it. Any time static assets change that don't require any additional tasks, we run livereload immediately.
Even better? When the CSS changes, there's not even a need for a browser refresh. The styles are automatically injected. It's brilliant.
grunt serve:dist
Running our environment locally is great. But when we do different things for local development and deployments, it's a pain to find out later that something went wrong. So we made a task that runs our local environment off the same files we'd be deploying. It won't necessarily fix all of our bugs, but it sure helps out a lot!
You might remember when we first went over our grunt serve task above, you probably noted the if statement that was included:
if (target === 'dist') {
return grunt.task.run(['build', 'connect:dist:keepalive']);
}
This statement allows us to do different actions with our grunt serve:dist task.
We've removed all the other tasks and modified our connect task to only serve content from our dist directory. The keepalive option is something that can be appended ad-hoc to connect task calls. Here's that modified connect task options Object:
options: {
open: true,
middleware: function (connect) {
var proxy = require('grunt-connect-proxy/lib/utils').proxyRequest;
return [
// Include the proxy first
proxy,
modRewrite(['!/api|\\.jpg|\\.gif|\\png|\\.svg|\\.woff2|\\.eot|\\.html|\\.js|\\.css|\\.woff|\\.ttf|\\.swf$ /index.html [L]']),
// Serve static files.
connect.static(appConfig.dist)
];
}
}
Wrapping up our local development process
To recap, we just automated a lot of our tasks! Reloading the page, compiling our Sass files, injecting dependencies are just a few of the options you can do with Grunt while you're coding. You could compile webfonts, generate spritesheets from your images, or even go crazy and create a statically-generated site with grunt-jekyll or assemble.
To help you combine everything that's above, we've created a gist with our entire Gruntfile.js and package.json.
Have any questions or comments? Did we miss something in our guide? We'd be delighted to read and answer them!
Related Posts
