A PHP Developer in The JavaScript World
Table of contents |
---|
Briefly
I want to know how to setup a JavaScript project in a proper way. In this article we create a simple form with a file upload, email sending with attachments and a MySQL database connection. We'll use Node.js, Express, Grunt, Jasmine, Bower, Bootstrap and many other libraries.
- 2021-12-05 Grammar is checked.
- 2018-01-05 Added recommendations of more modern technologies when appropriate: nvm, hapi.js, Webpack, pm2.
- 2014-09-23 Text is published.
- 2014-09-19 Text is started.
Motivation
In 2014 my background is LAMP (Debian, Apache, MySQL/PostgreSQL, PHP) plus some JavaScript (jQuery, Backbone).
Because the web developing world is shifting towards event-based programming (reference), I want to take a better look at the JavaScript (aka JS).
My objective today is to create a simple web form using AngularJS and Node.js. I want to have both development and production environments. The workflow to build the versions must be easy job to do. In addition I want unit and end-to-end tests.
All my current projects' data is in MySQL and PostgreSQL databases so I want to see how to do that with JS.
Why angular? Because it's leading the frontend JS framework battle (reference).
While reading articles about angular, nodejs etc. I found an endless number of different components and modules like npm, grunt, express, grunt-express, bower and grunt-bower. It's not trivial to find out what we really need. In addition there are MEAN (reference) and Yeoman (reference) which are offering everything in one shot but I guess it's a good idea to build the stack once by yourself.
The project source codes are available (reference) but the commits are not strictly following the progress of this article.
Debian
First we need a development environment. I have a Debian Squeeze (6.0) box.
$ sudo mkdir /var/www/simpleform
$ sudo chown username:username /var/www/simpleform
$ chmod -R 755 /var/www/simpleform/
Node.js and npm
Node.js (reference) makes it possible to run JS applications in the backend (server-side). Many of the components in this project require it. We also want to get npm to be able to easily install other node modules (reference).
UPDATE (2018): Check out nvm (reference).
It's recommended to fetch nodejs directly from the repository and compile it by yourself because the software is evolving rapidly. So first install Git if you didn't already have it (reference).
Then change your working directory to where you can download the source code files of nodejs.
$ cd /opt
/opt
seems to be a preferrable (and ancient) way to store 3rd party software's source code (reference) but you can use your home directory as well. It's just a temporary place for the source code files. The actual binaries are generated in latter steps.
# download source code files
$ git clone https://github.com/joyent/node.git
# list versions
$ git tag
# currently the latest stable version is 0.10.32 (reference)
$ git checkout v0.10.32
$ ./configure
$ make
$ sudo make install
# test
$ node -v; npm -v
v0.10.32
1.4.28
Next we create a package.json
file which contains the meta data of the application.
$ cd /var/www/simpleform
$ npm init # creates package.json
Express
UPDATE (2018): Check out hapi.js (reference).
Express is a nodejs framework and in this project it will be the platform for the backend (server) codes. It gives helpers for things like MySQL connection and email sending.
# find out the latest stable version
$ npm info express version
$ npm install express@4.9.3 --save
The --save
flag adds the module to the package.json
. The file format is quite comprehensive but ^4.9.3
means all the versions after 4.9.3 and before 5.0.0 (reference).
Then we can write the backend (server) application.
$ vim server.js # set the filename in the package.json's "main"
var express = require('express');
var app = express();
app.get('/', function(req, res) {
res.send('Omenapa ovela');
});
var server = app.listen(3000, function() {
console.log('Listening on port %d', server.address().port);
});
After saving the file and running the application we should be able to see the Omenapa ovela text in http://yourdomain.com:3000
.
# run the backend (server) application
$ node server.js
nodemon
When developing the backend application I don't want to manually restart the application everytime I modify the source code. That's why we need the nodemon module (reference). Let's install it. After that we run the application by it.
$ npm install nodemon --save
$ nodemon server.js
We can test it by running the command in one terminal and edit the source code in another terminal. The application should restart automatically when the file is saved.
Bower
UPDATE (2018): bower is not needed anymore. npm handles both frontend and backend libraries.
Another package manager, after npm, is needed for the front-end (reference): Bower (reference). Install and initialize it.
$ npm install bower --save
$ bower init # creates bower.json
AngularJS
Finally it's time to install angular. A --save
flag adds the component to the bower.json
.
$ bower install angular --save
Next we make angular working on the page. Create an index.html
and update the server.js
. We also need a new node module: path (reference).
$ vim index.html
<doctype html>
<html ng-app>
<head>
<script src="angular/angular.min.js"></script>
</head>
<body>
<p>Your name: <input type="text" ng-model="name"></p>
<p>Hello !</p>
</body>
</html>
$ vim server.js
var express = require('express');
var app = express();
var path = require('path');
var router = express.Router();
app.use(router);
router.use(express.static(path.join(__dirname, 'bower_components'))); // location of angularjs
router.use(function(req, res){
res.sendFile(path.join(__dirname, '/index.html'));
});
var server = app.listen(3000, function() {
console.log('Listening on port %d', server.address().port);
});
$ npm install path --save
Then we should have a working angular page in http://yourdomain.com:3000
. There's an input field where you can write your name which is displayed on the page in real time. This is called two way data binding.
And because we are going to handle all the application routes in AngularJS we don't need any other view files in Express except the landing page: index.html
. The discussion of route conventions is still open (reference). The current suggestion uses #!
notation.
Bootstrap
For user interface we can use Bootstrap (reference).
$ bower install bootstrap
And then we can add some candy to the page. That's needed because I want to learn how to compress and minify several CSS and JS files.
$ vim index.html
<html lang="en" ng-app>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link href="bootstrap/dist/css/bootstrap.min.css" rel="stylesheet">
<link href="custom.css" rel="stylesheet">
</head>
<body>
<div class="container">
<div class="row">
<h1 class="text-center">Ciao !</h1>
</div>
<div class="row">
<div class="col-sm-4 col-sm-offset-4">
<form role="form">
<div class="form-group">
<div class="row">
<div class="col-sm-9">
<label for="name">Your name</label>
<input type="text" ng-model="name" class="form-control" id="name">
</div>
<div class="col-sm-3">
<span ng-show="name" class="glyphicon glyphicon-ok nameGiven"></span>
</div>
</div>
</div>
</div>
</form>
</div>
</div>
<script src="angular/angular.min.js"></script>
<script src="jquery/dist/jquery.min.js"></script>
<script src="bootstrap/dist/js/bootstrap.min.js"></script>
</body>
</html>
$ vim bower_components/custom.css # strange directory, but don't worry
.nameGiven {
color: green;
font-size: 50px;
}
File structure
Okay, now we should have enough files to think what could be an optimal file structure. At the moment the application's business logic is divided to just two files (server.js
, index.html
) which both locate in the root directory. In addition we have one extra CSS file (bower_components/custom.css
).
But because the application will expand, we want to make a clear separation between the client (frontend) and server (backend) files. We will also have three environments - that's why we need to create dev
, dist
and test
directories. In addition we need own directory for the actual source codes (src
) as well as for the data (data
). spec
and coverage
will be needed in unit testing and tmp
is a temporary working directory.
Now, go and reorganize your current three files (server.js
, index.html
, custom.js
) according to the new structure.
bower_components/ # frontend modules installed by bower
angular/
angular.js
angular.min.js
...
bootstrap/
...
bower.json # bower settings and dependencies
coverage/ # code coverage report files
data/ # json:s etc.
dist/ # a distribution (release, build) to be deployed in production/staging/testing
dev/ # a running development version
Gruntfile.js # grunt settings (will be created later)
node_modules/ # backend modules installed by npm
express/
nodemon/
...
package.json # backend (server) settings and dependencies
spec/
unit/
client/ # unit tests for client
controllers.js
server/ # unit tests for server
server.spec.js
e2e/
data/ # sample data for tests
src/ # the actual editable source code files of the application
client/ # client, frontend, ui
index.html # landing page (later index.jade)
css/
custom.css
js/ # angular logics
main.js
controllers.js
...
partials/ # view partials
form.html
...
server/ # server, backend
config.sample # a skeleton for config
server.js
test/ # a running version for tests
tmp/ # a working directory for Grunt
After reorganizing the files we have to update the paths in server.js
and index.html
. To save some internet space, I list only the relevant lines here.
router.use(express.static(path.join(__dirname, '../../bower_components')));
router.use(express.static(path.join(__dirname, '../client')));
...
res.sendFile(path.join(__dirname, '../client/index.html'));
<link href="css/custom.css" rel="stylesheet">
After these changes we must run the server from the src
directory.
$ nodemon src/server/server.js
Using git is not in the scope of this article but it should be enough to add only bower.json
, package.json
, Gruntfile.js
, src/
, data/
and spec/
there. There are counterarguments though (reference). The installation specific config
files must be manually written when a new installation is created but more about this in the next section.
Environments
As I wrote in the Introduction section, we are trying to mimic an ideal real world software project where we have different settings for different environments. We might already have created a few directories (dev
, dist
, test
) for different environments but at the moment we are running the software directly from the src
directory which is not good. But before going into the details of this section, we need to clarify the terms we are using.
By development I mean the environment that the developer is using during the developing process. This is where the debug flag is on, logging is all-inclusive and the JS and CSS files are in human readable format. In an ideal situation there are also unit tests which are run automatically after every development version built.
When the developer is ready (and has run the tests), he makes a distribution (sometimes called as build or release) to be deployed for other people to try out the software. Ideally there is a testing environment where only a limited group of trusted people can test the software (alpha testing). To ease the debugging it's usually a good idea to not have the files minified in the testing environment. When the found bugs are fixed, a new distribution (release candidate) is built and now the CSS and JS files are minified in order to decrease the number of HTTP requests and save bandwidth. The new distribution is deployed in a staging environment where everybody can play with the software (beta testing). And then a final distribution (often called as a stable) is built for the production (or live) environment. In case you don't have the staging phase, it's better to minify the files already for the testing environment to prevent surprises in production.
Be careful to not mix up test and testing environments. The test environment exists only for a short moment when the automatic tests are run. On the other hand testing is a real installation where other people can test the software.
Each of the environments (or installation to be exact) needs own settings. For instance the credentials to databases and other systems, as well as host names, ports, email settings and hashing salts differ. Also an usual practice is to add some obtrusive notification telling if we are in the development environment to make a clear visual separation to prevent human accidents when a tester didn't know he was actually using live data.
So, what we need are three directories in the file structure: one for a running development version (dev
), one for a built distribution (dist
) and one for automatic tests (test
). In addition we need configuration files for each installation.
Because this software project is so simple, we skip testing and staging environments and run the production version directly from the dist
directory. In a bigger project we would copy the distribution to somewhere else. Each installation would probably have own running directory where the distribution would be copied directly to or transfered via version control system.
Because we want to visually separate the environments we need a configurable env
variable and also different landing pages (index.html
) because that's the place where we link to your CSS and JS files.
So, first we create the config
file. The config.sample
is by the way just a skeleton file (containing empty variables) for the repository because the actual config contains installation specific settings and thereby cannot be in the repository. There could be also a config.default
to have default values which the actual config file extended. In this project we are going to need three config files: dev/server/config
, dist/server/config
and test/server/config
. But at this very point we only use one version because we don't fully support the different environments yet.
$ vim src/server/config
var config = {};
config.env = 'dev'; // dev/prod/test/testing/staging/...
config.host = 'localhost';
config.port = 3001; // 3000 for prod, 3002 for test
config.basePath = '/var/www/simpleform/';
config.dataPath = config.basePath + 'data/';
config.mysql = {};
config.mysql.host = 'localhost';
config.mysql.database = '';
config.mysql.user = '';
config.mysql.password = '';
config.gmail = {};
config.gmail.username = '';
config.gmail.password = '';
config.email = {};
config.email.fromName = '';
config.email.fromEmail = '';
config.email.to = '';
config.email.subject = '';
module.exports = config;
And then we create the index.html
files. Luckily we don't have to manually maintain the files because we can utilize Jade (reference), a nodejs templating language which supports conditions. After installating it we need to modify the index.html
quite a bit.
$ npm install jade --save
$ mv src/client/index.html src/client/index.jade
$ vim src/client/index.jade
doctype html
html(lang="en", ng-app)
head
meta(charset="utf-8")
meta(http-equiv="X-UA-Compatible", content="IE=edge")
meta(name="viewport", content="width=device-width, initial-scale=1")
link(href="bootstrap/dist/css/bootstrap.min.css", rel="stylesheet")
link(href="css/custom.css", rel="stylesheet")
body
.container
.row
if is_prod
h2(class="text-center bg-danger") Production
else if is_test
h2(class="text-center bg-danger") Test
else
h2(class="text-center bg-success") Development
.row
h1(class="text-center") Ciao !
.row
div(class="col-sm-4 col-sm-offset-4")
form(role="form", name="simpleform")
.form-group
.row
.col-sm-9
label(for="name") Your name
input(type="text", ng-model="name", class="form-control", id="name")
.col-sm-3
span(ng-show="name", class="glyphicon glyphicon-ok nameGiven")
script(src="angular/angular.min.js")
script(src="jquery/dist/jquery.min.js")
script(src="bootstrap/dist/js/bootstrap.min.js")
Then we can generate the index files.
$ jade -P -O "{is_prod: true}" < src/client/index.jade > src/client/index_prod.html # P = pretty html
$ jade -P -O "{is_dev: true}" < src/client/index.jade > src/client/index_dev.html # O = options
$ jade -P -O "{is_test: true}" < src/client/index.jade > src/client/index_test.html
We also need to modify the server.js
to get it supporting the environments.
$ vim src/server/server.js
var express = require('express');
var app = express();
var path = require('path');
var router = express.Router();
app.use(router);
var config = require('./config');
router.use(express.static(path.join(__dirname, '../../bower_components')));
router.use(express.static(path.join(__dirname, '../client')));
router.use(function(req, res) {
if (config.env == 'prod') {
res.sendFile(path.join(__dirname, '../client/index_prod.html'));
} else if (config.env == 'test') {
res.sendFile(path.join(__dirname, '../client/index_test.html'));
} else {
res.sendFile(path.join(__dirname, '../client/index_dev.html'));
}
});
var server = app.listen(config.port, function() {
console.log('Listening on port %d', server.address().port);
});
Ok, so now we are supporting the environments: dev
, prod
and test
. Unfortunately we still have to manually generate the index files but don't worry, we will get back to it. But now we should see a notification bar telling the environment depending on what we have in src/server/config
(env
variable). You need to restart the server to see the change on the page.
As you might guess the production version is running in http://yourdomain.com:3000
, the development version in port 3001 and the testing version in port 3002 (or whatever you wrote in the config file). In real world we wouldn't use weird ports but had domains like http://dev.youdomain.com
and http://yourdomain.com
. We would need a nginx web server to achieve that (reference).
Grunt
UPDATE (2018): Check out Webpack (reference).
In the Introduction section I also mentioned that the workflow of developing and building different versions of the application must be easy job to do. For this we have Grunt (reference). Gulp is an alternative (reference).
Grunt is a task runner to perform various kind of repetitive tasks such as concatenating, validating and minimizing files, running tests and creating distributions. We need to install both grunt command line interface (CLI) and a local grunt task runner.
$ npm install -g grunt-cli --save-dev
$ npm install grunt --save-dev
A --save-dev
flag is used because we want grunt to go under the devDependencies
in package.json
because grunt is not needed in production.
Moreover if you haven't created dev/server/config
, dist/server/config
and test/server/config
yet, do so now. Copy the src/server/config
and modify the env
and port
variables.
Gruntfile.js
Let's first install all the grunt modules that we are going to need. It's recommended to use only officially maintained grunt-contrib-*
modules (reference).
$ npm install grunt-contrib-clean --save-dev
$ npm install grunt-contrib-jade --save-dev
$ npm install grunt-contrib-jshint --save-dev
$ npm install grunt-contrib-copy --save-dev
$ npm install grunt-contrib-watch --save-dev
$ npm install grunt-contrib-concat --save-dev
$ npm install grunt-contrib-uglify --save-dev
$ npm install grunt-contrib-cssmin --save-dev
$ npm install grunt-contrib-jasmine --save-dev
$ npm install grunt-jasmine-node --save-dev
$ npm install grunt-template-jasmine-istanbul --save-dev
Grunt tasks are configured, loaded and registered in the Gruntfile.js
file. Let's create it. There are a lot of stuff which we don't need right now but later on so you can ignore the lines you don't understand.
$ vim Gruntfile.js
module.exports = function(grunt) {
var devConfig = require('./dev/server/config');
var testConfig = require('./test/server/config');
var prodConfig = require('./dist/server/config');
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
clean: {
tmp: ["tmp/*"],
dev: [
"dev/*",
"!dev/server", // ! = exclude
"!dev/server/config", // installation specific, never deleted
"!dev/server/server.js" // because of nodemon, we need to overwrite the file, not delete and copy
],
dist: [
"dist/*",
"!dist/server",
"!dist/server/config"
],
test: [
"test/*",
"!test/server",
"!test/server/config"
]
},
jade: {
dev: {
options: {
pretty: true,
data: {is_dev: true, host: devConfig.host}
},
files: {"tmp/index.html": "src/client/index.jade"}
},
prod: {
options: {
pretty: true,
data: {is_prod: true}
},
files: {"tmp/index.html": "src/client/index.jade"}
},
test: {
options: {
pretty: true,
data: {is_test: true}
},
files: {"tmp/index.html": "src/client/index.jade"}
}
},
jshint: {
files: [
'Gruntfile.js',
'src/client/**/*.js',
'src/server/**/*.js',
'spec/**/*.js'
]
},
copy: {
dev: {
files: [
{src: 'src/server/server.js', dest: 'dev/server/server.js'},
{src: 'tmp/client.concat.js', dest: 'dev/client/js/client.concat.js'},
{src: 'tmp/client.concat.css', dest: 'dev/client/css/client.concat.css'},
{expand: true, cwd: 'bower_components/bootstrap/', src: 'fonts/**', dest: 'dev/client/'},
{src: 'tmp/index.html', dest: 'dev/client/index.html'},
{expand: true, cwd: 'src/client/', src: 'partials/*', dest: 'dev/client/'}
]
},
dist: {
files: [
{src: 'src/server/server.js', dest: 'dist/server/server.js'},
{src: 'tmp/client.min.js', dest: 'dist/client/js/client.min.js'},
{src: 'tmp/client.min.css', dest: 'dist/client/css/client.min.css'},
{expand: true, cwd: 'bower_components/bootstrap/', src: 'fonts/**', dest: 'dist/client/'},
{src: 'tmp/index.html', dest: 'dist/client/index.html'},
{expand: true, cwd: 'src/client/', src: 'partials/*', dest: 'dist/client/'}
]
},
test: {
files: [
{src: 'src/server/server.js', dest: 'test/server/server.js'},
{src: 'tmp/client.min.js', dest: 'test/client/js/client.min.js'},
{src: 'tmp/client.min.css', dest: 'test/client/css/client.min.css'},
{expand: true, cwd: 'bower_components/bootstrap/', src: 'fonts/**', dest: 'test/client/'},
{src: 'tmp/index.html', dest: 'test/client/index.html'},
{expand: true, cwd: 'src/client/', src: 'partials/*', dest: 'test/client/'}
]
}
},
watch: {
files: [
'Gruntfile.js',
'src/**',
'test/**'
],
tasks: ['dev'],
options: {
livereload: 35729,
spawn: false // don't create a new process if one is already running
}
},
concat: {
js_app: {
src: [
'src/client/js/main.js',
'src/client/js/controllers.js',
'src/client/js/services.js',
'src/client/js/directives.js'
],
dest: 'tmp/app.concat.js'
},
js_vendor: {
src: [
'bower_components/angular/angular.js',
'bower_components/angular-ui-router/release/angular-ui-router.js',
'bower_components/angular-mocks/angular-mocks.js',
'bower_components/jquery/dist/jquery.js',
'bower_components/bootstrap/dist/js/bootstrap.js',
],
dest: 'tmp/vendor.concat.js'
},
js: {
src: [
'tmp/vendor.concat.js',
'tmp/app.concat.js'
],
dest: 'tmp/client.concat.js'
},
css: {
src: ['bower_components/**/*.css', 'src/**/*.css'],
dest: 'tmp/client.concat.css'
}
},
uglify: {
files: {src: ['tmp/client.concat.js'], dest: 'tmp/client.min.js'}
},
cssmin: {
files: {src: ['tmp/client.concat.css'], dest: 'tmp/client.min.css'}
},
jasmine: {
files: {src: 'tmp/app.concat.js'},
options: {
vendor: 'tmp/vendor.concat.js',
specs: 'spec/unit/client/*.js',
template: require('grunt-template-jasmine-istanbul'),
templateOptions: {
coverage: 'coverage/coverage.json',
report: 'coverage'
}
}
},
jasmine_node: {
server: ['spec/unit/server/']
}
});
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-contrib-jade');
grunt.loadNpmTasks('grunt-contrib-jshint');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-uglify');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-copy');
grunt.loadNpmTasks('grunt-contrib-jasmine');
grunt.loadNpmTasks('grunt-jasmine-node');
grunt.registerTask('dev', [
'clean:tmp',
'jade:dev',
'jshint',
'concat_all',
'clean:dev',
'copy:dev',
'watch'
]);
grunt.registerTask('prod', [
'clean:tmp',
'jade:prod',
'jshint',
'concat_all',
'uglify',
'cssmin',
'clean:dist',
'copy:dist'
]);
grunt.registerTask('test', [
'clean:tmp',
'jade:test',
'jshint',
'concat_all',
'uglify',
'cssmin',
'clean:test',
'copy:test',
'jasmine',
'jasmine_node:server'
]);
grunt.registerTask('concat_all', [
'concat:js_app',
'concat:js_vendor',
'concat:js',
'concat:css',
]);
};
It's recommended to backup the source code files before running the tasks because in the worst case grunt can delete all the files.
So, we have three grunt tasks: grunt dev
, grunt prod
and grunt test
. Before analyzing what's happening when running them, we need to make a few file modifications. Replace the current JS and CSS file links by the lines below.
$ vim src/client/index.jade
if is_dev
link(href="css/client.concat.css", rel="stylesheet")
else
link(href="css/client.min.css", rel="stylesheet")
if is_dev
script(src="js/client.concat.js")
script(src="//#{host}:35729/livereload.js")
else
script(src="js/client.min.js")
Also because from now on the index.html
:s are generated to own environment directories, we don't need to separate them in server.js
anymore. Also one line can be removed.
$ vim src/server/server.js
router.use(function(req, res) {
res.sendFile(path.join(__dirname, '../client/index.html'));
});
router.use(express.static(path.join(__dirname, '../../bower_components'))); // remove this line
grunt dev
$ grunt dev
grunt dev
runs the following task respectively: clean:tmp
, jade:dev
, jshint
, concat_all
, clean:dev
, copy:dev
and watch
. Each task can be also run individually like grunt clean:tmp
.
clean:tmp
empties thetmp
directory.jade:dev
generates anindex.html
withis_dev: true
parameter and saves the file to atmp/client
directory.jshint
validates the JS files.concat_all
runs theconcat_all
task which runs its sub-tasks.concat:js_app
concatenates the application JS files.concat:js_vendor
concatenates the vendor JS files.concat:js
concatenates the application and the vendor JS files.concat:css
concatenates the CSS files.
clean:dev
empties thedev
directory preservingconfig
(installation specific data) andserver.js
(because of nodemon).copy:dev
copies all the relevant files to thedev
.server.js
is overwritten and thanks to nodemon it's automatically restarted.watch
watches the editable files and rerunsgrunt dev
if some of the files were changed. This is a really nice feature because we don't have to manually use grunt when developing.
So, let's edit and save some of the source files and see how grunt automatically validates the files and builds a new development distribution (don't worry if the server dies now - we'll fix it in a moment). We can actually go further by utilizing livereload (reference) to get the browser to automatically update the page when something is changed. Because we have already made the support for livereload in Gruntfile.js
and in index.jade
we only need to update the domain to the config
files host
parameter.
$ vim dev/server/config
config.host = 'yourdomain.com';
We also have to run the server from the dev
directory from now on. In addition we add a --watch
argument which makes nodemon to restart only if the dev/server/server.js
file is changed because we don't want to make it restart everytime when any of the project files is changed. Grunt with watch module handles the restarting procedure now.
$ nodemon --watch dev/server/server.js dev/server/server.js
Now go and try to edit something in the index.jade
. After saving the file we should see a miracle: the browser updates the page itself.
Now the development environment is ready. Because it's a bit inconvenient to have (at least) two terminals opened when developing, you can use screen's windows vertically split (reference).
$ screen -S simpleform # start a new screen with name simpleform
$ cd /var/www/simpleform; nodemon --watch dev/server/server.js dev/server/server.js
C-a c # create a new window (C = Ctrl)
C-a S # split screen
C-a [tab] # move to lower split
C-a [space] # change to next window
$ grunt dev
C-a d # detach the screen
grunt prod
Then let's analyze the procedure of generating the production distribution.
$ grunt prod
clean:tmp
empties thetmp
directory.jade:prod
generates anindex.html
withis_prod: true
parameter and saves the file to atmp/client
directory.jshint
validates the JS files.concat_all
runs theconcat_all
task (seegrunt dev
).uglify
minifies the client JS files.cssmin
minifies the client CSS files.clean:dist
empties thedist
directory preservingconfig
(installation specific data).copy:dist
copies the relevant files to thedist
.
As we see, there are many similarities between the dev and prod tasks. The main difference is that in production the CSS and JS files are minified when in development they are just concatenated to ease debugging.
If we had more environments such as staging and testing, we could create own tasks and directories for them as well. Grunt is actually even able to copy the files over SSH so it could be possible to have tasks like dist-testing
, dist-staging
and dist-production
which copied the files wherever needed. But the traditional approach to have own cloned repositories per installation might be even better because then transfering files and having version branches is managed in more flexible fashion. But in this project we run both environments in the same directory (/var/www/simpleform
).
forever
UPDATE (2018): Check out pm2 (reference).
In development we are using nodemon
, grunt-contrib-watch
and livereload.js
to ease the developing work but in production we have different needs.
First we definately don't want to update the production files after every edit we make. Secondly we want to control by ourselves when to restart the server. And thirdly the server has to be running in background endlessly, generating logs and restarting itself in case of system or network level error.
What we need to fulfill the requirements mentioned is forever (reference).
$ npm install forever --save
Start the server.
$ forever start --uid "simpleform" --append -l /var/log/simpleform/forever.log -o /var/log/simpleform/debug.log -e /var/log/simpleform/error.log dist/server/server.js
To restart the server manually, run the next commands. This is needed to be done when a new production distribution is deployed.
$ forever list
$ forever restart simpleform
If everything is right, we should see a production version running in the http://yourdomain.com:3000
.
grunt test
Finally we have a grunt task for the test environment which is analyzed next. The actual testing will be discussed later in the Testing section.
$ grunt prod
clean:tmp
empties thetmp
directory.jade:test
generates anindex.html
withis_test: true
parameter and saves the file to atmp/client
directory.jshint
validates the JS files.concat_all
runs theconcat_all
task (seegrunt dev
).uglify
minifies the client JS files.cssmin
minifies the client CSS files.clean:test
empties thetest
directory preservingconfig
(installation specific data).copy:test
copies the relevant files to thetest
.jasmine
runs the frontend unit tests and generates code coverage files to thecoverage
directory.jasmine_node:server
runs the backend unit tests.
As we don't have any tests written yet, there is no point to run the task. We will get back to this in the Testing section of the article.
MySQL
This section is skippable. We don't really need data from the MySQL database in this project but I wanted to try it for educational reasons. So, let's pretend that we have a users
table in the database and we want to serve the list in http://yourdomain.com:3001/users
as a JSON response.
Remember to add your MySQL credentials to the config files. Then just install a few node modules and add the necessary lines to server.js
.
$ npm install mysql --save
$ npm install express-myconnection --save
$ vim src/server/server.js
var mysql = require('mysql');
var myConnection = require('express-myconnection');
var config = require('./config');
app.use(myConnection(mysql, config.mysql, 'pool'));
router.use('/users', function(req, res, next) {
req.getConnection(function(err, connection) {
connection.query('SELECT * FROM users', function(err, results) {
if (err) {
return next(err);
}
return res.json(results);
});
});
});
router.use(function(req, res, next) {
res.sendFile(path.join(__dirname, '../client/index.html'));
});
The application
Finally it's time to start coding the business logic itself. We are going to build a form with a few basic static fields and some dynamically generated fields from a JSON data file. When the form is submitted, the data is emailed to an email address we configure.
Because this article is more about the stack than angular coding, we are not going into the details of the application logic.
Data
At first we create a JSON file. To save some internet space the file is included here as minified. Just paste it to JSONLint (reference) to make it human readable.
$ vim data/form1.json
[{"label":"What's your biggest concern?","id":"main-concern","name":"mainConcern","type":"select","options":[{"label":"Health","value":"health","nextFields":["health"]},{"label":"Money","value":"money","nextFields":["money"]},{"label":"Wife","value":"wife","nextFields":["nothing-to-do"]}]},{"label":"Oh damn, what's wrong with you?","id":"health","name":"health","type":"select","options":[{"label":"Back pain","value":"back-pain","nextFields":["sport-hours-per-week","health-backpain-submit"]},{"label":"Sanity","value":"sanity","nextFields":["age","poem","health-sanity-submit"]}]},{"label":"Poor man! What's the problem?","id":"money","name":"money","type":"select","options":[{"label":"I've got too little","value":"too-little","nextFields":["money-toolittle-submit"]},{"label":"I've got too much","value":"too-much","nextFields":["money-toomuch-submit"]}]},{"label":"Sorry, nothing to do.","id":"nothing-to-do","name":"nothingToDo","value":"period","type":"hidden"},{"label":"How many hours of sport you do per week?","id":"sport-hours-per-week","name":"sportHoursPerWeek","type":"radio","options":[{"label":"less than 5","value":"less-than-5"},{"label":"more than 5 or 5","value":"more-than-5"}]},{"label":"What's your age?","id":"age","name":"age","type":"text"},{"label":"Write a poem.","id":"poem","name":"poem","type":"textarea"},{"label":"","id":"health-backpain-submit","name":"submit","type":"submit"},{"label":"","id":"health-sanity-submit","name":"submit","type":"submit"},{"label":"","id":"money-toomuch-submit","name":"submit","type":"submit"},{"label":"","id":"money-toolittle-submit","name":"submit","type":"submit"}]
In addition we need the backend to return the data for the frontend.
$ vim src/server/server.js
var fs = require('fs');
router.get('/form/:id', function(req, res, next) {
var sanitizedId = +(req.params.id)
fs.readFile(config.dataPath + 'form' + sanitizedId + '.json', {encoding: 'utf8'}, function(err, data) {
return res.json(JSON.parse(data));
});
});
Then we get the data from http://yourdomain.com:3001/form/1
.
Static form
It's time for the angular code. But first we install yet another component, angular-ui-router, because internet recommends it (reference).
$ bower install angular-ui-router --save
As I told earlier, index.html
is just a landing page and angular handles the rest views and routes. So, let's modify it a little bit and move the form to src/client/partials/form.html
. Let's also add a few new static form input elements (Image, Description).
$ vim src/client/partials/form.html
<form role="form", ng-controller="FormController">
<h1 class="text-center">Ciao !</h1>
<div class="form-group">
<div class="row">
<div class="col-sm-9">
<label for="name">Your name</label>
<input type="text" ng-model="name" id="name" class="form-control">
</div>
<div class="col-sm-3">
<span ng-show="name" class="glyphicon glyphicon-ok nameGiven"></span>
</div>
</div>
</div>
<div class="form-group">
<label>Image</label>
<input type="file" name="file" class="form-control">
</div>
<div class="form-group">
<label>Description</label>
<textarea ng-model="description" name="description" class="form-control"></textarea>
</div>
</form>
$ vim src/client/index.jade
doctype html
html(lang="en", ng-app="simpleform")
head
meta(charset="utf-8")
meta(http-equiv="X-UA-Compatible", content="IE=edge")
meta(name="viewport", content="width=device-width, initial-scale=1")
if is_dev
link(href="css/client.concat.css", rel="stylesheet")
else
link(href="css/client.min.css", rel="stylesheet")
body
.container
.row
if is_prod
h2(class="text-center bg-danger") Production
else if is_test
h2(class="text-center bg-danger") Test
else
h2(class="text-center bg-success") Development
.row
div(class="col-sm-4 col-sm-offset-4")
div(ui-view)
if is_dev
script(src="js/client.concat.js")
script(src="//#{host}:35729/livereload.js")
else
script(src="js/client.min.js")
Next we create two JS files for the frontend: main.js
is where the frontend application module is created and in controllers.js
we have frontend controllers.
$ vim src/client/js/main.js
var app = angular.module('simpleform', ['ui.router']);
app.config(['$stateProvider', function($stateProvider) {
$stateProvider.state('front', {
url: "",
templateUrl: "partials/form.html",
controller: 'FormController'
});
}]);
$ vim src/client/js/controllers.js
app.controller('FormController', function($scope) {
function init() {}
init();
});
After this we have the form markup coming from the form.html
file served by a FormController
controller.
Dynamic fields
We have a form with three static fields at the moment. But we want to generate more fields dynamically from the JSON data.
$ vim src/client/partials/form.html
...
<textarea ng-model="description" name="description" class="form-control"></textarea>
</div>
<div class="form-group" ng-repeat="field in form" ng-show="field.show || $first">
<label></label>
<select ng-if="field.type == 'select'" ng-model="field.value" ng-change="showNextFields(field)" name="{{ field.name }}" id="{{ field.id }}" class="form-control">
<option value=""></option>
<option ng-repeat="option in field.options" value="{{ option.value }}"></option>
</select>
<input ng-if="field.type == 'text'" ng-model="field.value" type="text" name="{{ field.name }}" value="{{ field.value }}" id="{{ field.id }}" class="form-control">
<input ng-if="field.type == 'hidden'" ng-model="field.value" type="hidden" name="{{ field.name }}" value="{{ field.value }}" id="{{ field.id }}" class="form-control">
<textarea ng-if="field.type == 'textarea'" ng-model="field.value" name="{{ field.name }}" id="{{ field.id }}" class="form-control"></textarea>
<div ng-if="field.type == 'radio'" ng-repeat="option in field.options" id="{{ field.id }}" class="radio">
<label><input type="radio" ng-model="field.value" name="{{ field.name }}" value="{{ option.value }}"> </label>
</div>
<input ng-if="field.type == 'submit'" type="submit" name="{{ field.name }}" id="{{ field.id }}" value="Submit" class="btn btn-primary btn-lg btn-block">
</div>
$ vim src/client/js/controllers.js
app.controller('FormController', ['$scope', '$http', function($scope, $http) {
function init() {
$scope.form = [];
$http.get('/form/1').then(function(data) {
$scope.form = data.data;
});
}
$scope.showNextFields = function(field) {
var options = field.options;
var i, j, k = 0;
// first find field's nextFields
var nextFields = [];
for (i = 0; i < options.length; i++) {
if (options[i].value == field.value) {
if (typeof options[i].nextFields !== "undefined") {
nextFields = options[i].nextFields;
}
i = options.length;
}
}
// show the field itself and the nextStep fields but hide the rest
for (j = 0; j < $scope.form.length; j++) {
if ($scope.form[j].id == field.id) {
$scope.form[j].show = true;
} else {
$scope.form[j].show = false;
// show the nextStep fields
for (i = 0; i < nextFields.length; i++) {
if ($scope.form[j].id == nextFields[i]) {
$scope.form[j].show = true;
}
}
}
}
// show all the fields whose child field in nextField tree is shown
for (j = 0; j < $scope.form.length; j++) {
var maxIterations = 100;
if (isChildFieldShown($scope.form[j].id, maxIterations)) {
$scope.form[j].show = true;
}
}
// empty values of hidden fields
for (j = 0; j < $scope.form.length; j++) {
if ($scope.form[j].show === false) {
if (typeof $scope.form[j].value !== 'undefined') {
$scope.form[j].value = '';
}
}
}
};
function isChildFieldShown(fieldId, maxIterations) {
if (maxIterations-- < 0) {
return false;
}
var form = $scope.form;
var i, j, k = 0;
for (k = 0; k < form.length; k++) {
if (form[k].id != fieldId) {
continue;
}
if (form[k].show) {
return true;
}
if (typeof form[k].options !== "undefined" && form[k].options) {
for (j = 0; j < form[k].options.length; j++) {
if (typeof form[k].options[j].nextFields !== "undefined" && form[k].options[j].nextFields) {
for (i = 0; i < form[k].options[j].nextFields.length; i++) {
if (isChildFieldShown(form[k].options[j].nextFields[i], maxIterations)) {
return true;
}
}
}
}
}
}
return false;
}
init();
}]);
Now the form is generated dynamically from the JSON data.
Submit and file upload
Because the form has a file upload, we need a new module: multiparty (reference). We also need several file modifications.
$ npm install multiparty --save
$ vim src/client/partials/form.html
<form role="form" ng-submit="submit()" ng-controller="FormController">
...
<input type="file" name="file" class="form-control" file-model="file">
...
<input ng-if="field.type == 'submit'" type="submit" name="{{ field.name }}" id="{{ field.id }}" value="Submit" class="btn btn-primary btn-lg btn-block" ng-disabled="fileUploading">
$ vim src/client/js/controllers.js
app.controller('FormController', ['$scope', '$http', 'fileUpload', function($scope, $http, fileUpload) {
...
$scope.submit = function() {
$scope.fileUploading = true;
$scope.fields = [];
if (typeof $scope.description !== 'undefined') {
$scope.fields.push({description: $scope.description});
}
if (typeof $scope.name !== 'undefined') {
$scope.fields.push({name: $scope.name});
}
for (var i = 0; i < $scope.form.length; i++) {
if (typeof $scope.form[i].value !== 'undefined' && $scope.form[i].value !== '') {
var obj = {};
obj[$scope.form[i].name] = $scope.form[i].value;
$scope.fields.push(obj);
}
}
fileUpload.uploadFileToUrl('/answer', $scope.fields, $scope.file, function(response) {
$scope.fileUploading = false;
}, function(response) {
});
};
$ vim src/client/js/directives.js
app.directive('fileModel', ['$parse', function ($parse) {
return {
restrict: 'A',
link: function(scope, element, attrs) {
var model = $parse(attrs.fileModel);
var modelSetter = model.assign;
element.bind('change', function(){
scope.$apply(function(){
modelSetter(scope, element[0].files[0]);
});
});
}
};
}]);
$ vim src/client/js/services.js
app.service('fileUpload', ['$http', function ($http) {
this.uploadFileToUrl = function(url, fields, file, success, error){
var fd = new FormData();
for (var i = 0; i < fields.length; i++) {
for (var key in fields[i]) {
fd.append(key, fields[i][key]);
}
}
if (file) {
fd.append('file', file);
}
$http({
url: url,
method: "POST",
data: fd,
transformRequest: angular.identity,
headers: {'Content-Type': undefined}
}).then(
function(res) {success(res);},
function(res) {error(res);}
);
};
}]);
$ vim src/server/server.js
var multiparty = require('multiparty');
...
// route: form handler
router.post('/answer', function(req, res) {
var form = new multiparty.Form();
form.parse(req, function(error, fields, files) {
console.log(error, fields, files);
res.end();
});
});
Now the Submit button is making a request and the uploaded file is transfered to the /tmp/
directory in the server. Because of the console.log
in the server.js
you should see a debug message.
Emailing
Then we add emailing. Remember to update your Gmail credentials and email settings to the config files.
$ npm install nodemailer --save
$ vim src/server/server.js
var nodemailer = require('nodemailer');
...
// route: form handler
router.post('/answer', function(req, res) {
var form = new multiparty.Form();
form.parse(req, function(error, fields, files) {
var body = '';
for (var key in fields) {
body += key.toUpperCase() + ': ' + fields[key] + "\n";
}
var filesArr = typeof files.file !== 'undefined' ? files.file : [];
sendEmail(body, filesArr, function(info) {
if (filesArr && filesArr[0] && typeof filesArr[0].path !== 'undefined') {
fs.unlink(filesArr[0].path);
}
});
res.end();
});
});
var sendEmail = function(body, files, next) {
var transporter = nodemailer.createTransport({
service: 'Gmail',
auth: {
user: config.gmail.username,
pass: config.gmail.password
}
});
var attachments = [];
for (var i = 0; i < files.length; i++) {
attachments.push({
filename: files[i].originalFilename,
path: files[i].path
});
}
var emailOptions = {
from: config.email.fromName + ' <' + config.email.fromEmail + '>',
to: config.email.to,
subject: config.email.subject,
text: body,
attachments: attachments
};
transporter.sendMail(emailOptions, function(error, info){
if (error){
console.log(error, info);
next(info, error);
} else {
console.log('Message sent: ' + info.response);
next(info);
}
});
};
And the application is ready. The submitted data is emailed to the configured email address (config.email.to
).
Testing
One of the hardest part of web development is testing. Technically it's difficult to be on the higher abstraction level of the software which is usually needed for testing. The difficulty is highly dependant on the software design and modularity but even if the software is "perfectly" designed and implemented there are usually many integrations to other systems which need to be mocked and that takes time.
But even more crucial challenges are not technical but economical and related to risk analysis because in real world it's not reasonable or even possible to test everything (unless you are sending a spaceship to the moon).
The bottom line is to figure out what are the most important functions of the software and which of them are most likely to fail. In other words, what parts of the system has the best Return of Investment (aka ROI) to test?
There are code coverage tools like Istanbul (reference) which measure how large part of the software is executed when the tests are run. The code coverage tools serve some insights but the real software quality can't be evaluated only in quantitative manner because firstly some trivial codes are just waste of time to test. And secondly important and complex algorithms should be tested thoroughly in many different ways and inputs.
So, write tests for the most crucial and error-prone parts of the software and you quickly achieve 80 % satisfaction. To reach the rest 20 % would usually need too much resources. There's a name for this 80/20 idea: Pareto Principle.
In this project we will write unit tests both for the frontend and backend. In addition we will do end-to-end testing for the whole process: from the user interaction to check the email box.
If you haven't created the test/server/config
file yet, go and do it now. env
is "test" and port
could be 3002. The rest can be copied from dev/server/config
.
Unit testing (frontend)
To test the frontend we need a new module (what a surprise).
$ bower install angular-mocks --save
In the Grunt section of this article we installed Jasmine (reference) and Istanbul (reference).
Jasmine is a testing framework using a headless browser PhantomJS (reference) which can be run in console. The code coverage tool Istanbul can be integrated with Jasmine. There is also Mocha (reference).
It's also possible to use a test runner Karma (reference) with Jasmine if the tests are required to be run in real browsers (karma-cli, karma, karma-jasmine, karma-chrome-launcher, grunt-karma). However we are happy with PhantomJS which uses the same JS interpreter (WebKit) as Chrome (not exactly true) and Safari.
$ vim spec/unit/client/controllers.js
describe('Unit: FormController', function() {
beforeEach(module('simpleform'));
var ctrl, scope, httpBackend;
beforeEach(inject(function($injector) {
scope = $injector.get('$rootScope').$new();
createController = function() {
return $injector.get('$controller')('FormController', {$scope: scope });
};
httpBackend = $injector.get('$httpBackend');
// GET /form/1
var form = [{label:"What's your biggest concern?",id:"main-concern",name:"mainConcern",type:"select",options:[{label:"Health",value:"health",nextFields:["health"]},{label:"Money",value:"money",nextFields:["money"]},{label:"Wife",value:"wife",nextFields:["nothing-to-do"]}]},{label:"Oh damn, what's wrong with you?",id:"health",name:"health",type:"select",options:[{label:"Back pain",value:"back-pain",nextFields:["sport-hours-per-week","health-backpain-submit"]},{label:"Sanity",value:"sanity",nextFields:["age","poem","health-sanity-submit"]}]},{label:"Poor man! What's the problem?",id:"money",name:"money",type:"select",options:[{label:"I've got too little",value:"too-little",nextFields:["money-toolittle-submit"]},{label:"I've got too much",value:"too-much",nextFields:["money-toomuch-submit"]}]},{label:"Sorry, nothing to do.",id:"nothing-to-do",name:"nothingToDo",value:"period",type:"hidden"},{label:"How many hours of sport you do per week?",id:"sport-hours-per-week",name:"sportHoursPerWeek",type:"radio",options:[{label:"less than 5",value:"less-than-5"},{label:"more than 5 or 5",value:"more-than-5"}]},{label:"What's your age?",id:"age",name:"age",type:"text"},{label:"Write a poem.",id:"poem",name:"poem",type:"textarea"},{label:"",id:"health-backpain-submit",name:"submit",type:"submit"},{label:"",id:"health-sanity-submit",name:"submit",type:"submit"},{label:"",id:"money-toomuch-submit",name:"submit",type:"submit"},{label:"",id:"money-toolittle-submit",name:"submit",type:"submit"}];
httpBackend.when("GET", "/form/1").respond(form);
}));
afterEach(function() {
httpBackend.verifyNoOutstandingExpectation();
httpBackend.verifyNoOutstandingRequest();
});
it('should return error and leave fileUploading true',
function() {
expect(scope.fileUploading).not.toBeDefined();
var controller = createController();
httpBackend.expect("POST", "/answer").respond(500);
scope.submit();
httpBackend.flush();
expect(scope.fileUploading).toBe(true);
});
it('should return success and set fileUploading false',
function() {
var controller = createController();
httpBackend.expect("POST", "/answer").respond(200);
scope.submit();
httpBackend.flush();
expect(scope.fileUploading).toBe(false);
});
it('should handle form fields\' shows and values',
function() {
var controller = createController();
httpBackend.flush(); // GET /form/1
var poemFieldIdx, healthFieldIdx, moneyFieldIdx, mainConcernFieldIdx;
for (var i = 0; i < scope.form.length; i++) {
if (scope.form[i].id == 'main-concern') {
mainConcernFieldIdx = i;
} else if (scope.form[i].id == 'health') {
healthFieldIdx = i;
} else if (scope.form[i].id == 'money') {
moneyFieldIdx = i;
} else if (scope.form[i].id == 'poem') {
poemFieldIdx = i;
} else if (scope.form[i].id == 'sanity') {
sanityFieldIdx = i;
}
}
scope.form[healthFieldIdx].value = 'sanity'; // sanity option selected
scope.showNextFields(scope.form[healthFieldIdx]);
scope.form[poemFieldIdx].value = 'This is a poem';
scope.description = 'This is a description.';
scope.name = 'Lassi';
scope.submit();
expect(scope.form[mainConcernFieldIdx].show).toBe(true);
expect(scope.form[healthFieldIdx].show).toBe(true);
expect(scope.form[moneyFieldIdx].show).toBe(false);
expect(scope.form[poemFieldIdx].show).toBe(true);
httpBackend.expect("POST", "/answer").respond(200);
httpBackend.flush(); // POST /answer
expect(scope.fields[0].description).toBe('This is a description.');
expect(scope.fields[1].name).toBe('Lassi');
expect(scope.fields[2].health).toBe('sanity');
expect(scope.fields[3].poem).toBe('This is a poem');
});
});
The tests can be run by grunt test
command. The code coverage report files are generated into the coverage
directory. Open the directory in the browser to see some nice charts.
Unit testing (backend)
In backend unit tests we need a grunt-jasmine-node module which was already installed in Grunt section. It makes it possible to use Jasmine testing framework for the backend code.
$ mkdir spec/data; echo "some test data" > spec/data/sample1.txt
$ npm install request --save-dev
$ vim spec/unit/server/server.spec.js
var request = require('request');
var fs = require('fs');
var config = require('../../../test/server/config');
var server = require(config.basePath + '/test/server/server.js');
describe('server', function() {
var baseUrl = 'http://' + config.host + ':' + config.port;
it("should response form json", function(done) {
request.get(baseUrl + '/form/1', function(error, response) {
expect(response.body).toMatch(/nextFields":\["money\-toolittle\-submit"\]\},\{"label":"I\'ve got too much/);
done();
});
});
it("should response index page", function(done) {
request.get(baseUrl + '/', function(error, response) {
expect(response.body).toMatch(/Test/);
expect(response.body).toMatch(/div ui-view/);
done();
});
});
it("should email form data", function(done) {
var formData = {
name: 'Matti Kutonen',
description: 'Dataa',
something: 'else',
file: fs.createReadStream(config.basePath + 'spec/data/sample1.txt'),
};
request.post({url: baseUrl + '/answer', formData: formData}, function(error, response) {
done();
});
});
});
And again the tests are run by the grunt test
command.
End-to-end testing
End-to-end testing means testing the whole application logic. In the application it means opening the page, selecting and filling the form fields, submitting the form and finally checking the email box. Many sources suggest using Protractor which ought to be perfect for angular projects (reference).
WARNING: Don't try this at home.
Protractor requires Selenium WebDriver (reference) which I've had troubles with a few times in the past and therefore I wasn't very eager to install it. But after reading that angular team recommends Protractor with Selenium I was encouraged enough to try it one more time. So let's install it and start the selenium.
$ npm install protractor
$ ./node_modules/protractor/bin/webdriver-manager update
$ ./node_modules/protractor/bin/webdriver-manager start
...
throw er; // Unhandled 'error' event
...
Promising. A very good error report. Let's google. Aha, we need some JDK installed. No mention about this in the official tutorial on Protractor page. Let's install it. Because I have Debian 6.0 the default version for JDK is 6 when 7 is the latest stable.
$ sudo apt-get install openjdk-6-jdk
Then we copy the sample spec and config files from the Protractor page.
$ vim todo-spec.js
describe('angularjs homepage todo list', function() {
it('should add a todo', function() {
browser.get('http://www.angularjs.org');
element(by.model('todoText')).sendKeys('write a protractor test');
element(by.css('[value="add"]')).click();
var todoList = element.all(by.repeater('todo in todos'));
expect(todoList.count()).toEqual(3);
expect(todoList.get(2).getText()).toEqual('write a protractor test');
});
});
$ vim conf.js
exports.config = {
seleniumAddress: 'http://localhost:4444/wd/hub',
specs: ['todo-spec.js']
};
And then we can test it.
$ ./node_modules/protractor/bin/protractor conf.js
...
error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory
21:05:18.551 ERROR - org.apache.commons.exec.ExecuteException: Process exited with an error: 127 (Exit value: 127)
...
[launcher] Error: UnknownError: null
....
I hate Selenium. I knew this. All the other modules and components during the process of making this article have been working like charm but this is something special. Really special. Let's google.
We seem the be missing libconf2-4
. Come again? Do we really need to install this scary looking library to get angular e2e tests working? I'm not a sysadmin but what I know is that you should be really cautious with these kind of things because you can get your system messed up if you don't know what you are doing. And I don't know what I'm doing. But you don't learn without trying so let's install it and rerun the test.
$ sudo apt-get install libgconf2-4
$ ./node_modules/protractor/bin/protractor conf.js
/usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by chromedriver)
/usr/lib/libnss3.so: version `NSS_3.14.3' not found (required by chromedriver)
21:28:04.769 ERROR - org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
ARGARHARGGHRAHHHRAHR. Enough. Burn in hell. No e2e testing this time.
Final words
The project is completed. We built a small form which submits data to an email box. There are three environments (dev
, prod
, test
) which can be used efficiently. The final source codes are available (reference).
End-to-end testing is something we didn't achieve eventhough it was set as a goal. The reason to fail with it is maybe too old Debian version (6.0) but then again, how this can be so difficult? Perhaps it's because Protractor uses native browsers (which is nice). It's also possible to do the testing with PhantomJS, for example, but it doesn't use browsers natively. UPDATE (2023): Playwright.
I would have liked to run the application in sub-domains without the ugly ports in the URL but because it isn't apparently possible to do without nginx (or other web server) I skipped it.
What we learnt are the basics of AngularJS as well as Grunt, Bootstrap, nodejs, unit testing with Jasmine, npm and Bower. If I kept on trying new things I would check out MongoDB, Batarang, Restangular, Ember, React, Gulp, Yeoman. UPDATE (2018): Today I recommend React, MongoDB, Webpack and hapi.js.
I'm a bit worried what will happen with all the different modules and their dependencies when the time goes on. After a year or two I will probably want to update Angular or nodejs but if some of the core modules of the project has not been updated, the angular update can get nasty. We will see.