Node Angular Template

More for myself than anything, I created a base application template and put it up on GitHub. Feel free to fork it, and contributions are welcome. Here's what it has out of the box:

  • AngularJS 1.5
  • UI-Router
  • Bootstrap (LESS compiled)
  • Gulp Build
    • Lint
    • Transpile (ES6 -> ES5 with Babel)
    • Browserify
    • LESS
    • JS and CSS concatenation and minification
    • JS and CSS file versioning
    • and More!
  • JavaScript Unit Testing with Jasmine
  • Simple Node Server Script
  • Sample (Simple) AngularJS App with State Routing

I just needed something as a base to start other projects off of, so I threw this together as a starting point. You need Bower and NodeJS to get started, but all of the instructions are on the GitHub Page. Use it for your own projects, a quick look at AngularJS with ES6, or a sample of writing Gulp tasks.

All feedback encouraged and welcome.

ES2015 and Fun With Parameters

If you've come to JavaScript after learning to program in other languages, one thing that's probably stuck in your craw over the years has been the lack of any way to define default parameters in functions. You've probably written something like this in the past:

view plain print about
1var foo = function (bar) {
2        bar = bar || 'ok';
3        // ...
4    };

For the most part that sorta thing probably worked out, until your argument was a boolean, which then really complicated things.

With ES2015, the heavens have opened and prayers have been answered, as we've finally been given the ability to define default parameters. Consider the following:

view plain print about
1// This is in a class
2    foo (bar=true) {
3        if (bar) {
4            console.log('First round is on Simon!');
5        } else {
6            console.log('No drinks today!');
7        }
8    }

Simple. We're saying that if bar is true, then Simon is buying, otherwise we're out of luck, and defaulting our argument to true. We can then call this method a few times to test it out:

view plain print about
1constructor () {
2        this.foo(); // 'First round is on Simon!'
3        this.foo(false); // 'No drinks today'
4        let bar;
5        this.foo(bar); // 'First round is on Simon!'
6    }

You can see from my comments what the output of those methods would be. It's important to note here that even the undefined value passed as an argument triggered the default argument value as well.

Hopefully default arguments will help you to significantly simplify your code in the future. As usual, if you have any feedback/praise/complaints/war stories, feel free to comment below, or drop me a private message through the "contact" link on the page.

ES2015, Promises, and Fun With Scope

I've been using Promises for some time now. JQuery has acted as a shim for some time, and several other libraries have been around as well. ES2015 includes Promises natively. If you're unfamiliar with Promises, I strongly suggest you read this great post by Dave Atchley.

Like I said though, I've been using Promises for a while now. So, when I started moving to ES2015 it was a bit of a kick in the pants to find issues with implementing my Promises. Let me give you an example of how something might've been written before:

view plain print about
1'use strict';
2    
3    module.exports = [
4        '$scope', 'orders', 'dataService',
5        function ($scope, orders, dataService) {
6            var self = this;
7            
8            self.orders = orders;
9            
10            self.addOrder = function (order) {
11                // ... do some stuff
12                // get original
13                dataService.get(order.id)
14                    .then(self._updateOrders)
15                    .catch(function (error) {
16                        // do something with the error
17                    });
18            };
19            
20            // faux private function, applied to 'this' for unit testing
21            self._updateOrders = function (order) {
22                // ... some process got our order index from orders, then...
23                orders[index] = $.extend(true, orders[index], order);
24            };
25        }
26    ];

Seems pretty straightforward, right? addOrder() gets called, which does some stuff and then retrieves an order from the service. When the service returns the order, that's passed to the _updateOrders() method, where it finds the correct item in the array and updates it (I know, it's flawed, but this is just an example to show the real problem).

So, what's the problem? That works great. Has for months (or even years). Why am I writing this post? Fair question. Let's take a look at refactoring this controller into an ES2015 class. Our first pass might look like this:

view plain print about
1'use strict';
2    
3    class MyController {
4        constructor ($scope, orders, dataService) {
5            this.$scope = $scope;
6            const myOrders = [];
7            this.orders = myOrders.push(orders);
8            this._dataService = dataService;
9        }
10        
11        addOrder (order) {
12            // ... do some stuff
13            // get original
14            this._dataService.get(order.id)
15                .then(this._updateOrders)
16                .catch(function (error) {
17                    // do something with the error
18                });
19        }
20        
21        _updateOrders (order) {
22            // ... some process got our order index from orders, then...
23            this.orders[index] = $.extend(true, this.orders[index], order);
24        }
25    }
26    
27    MyController.$inject = ['$scope', 'orders', 'dataService'];
28    
29    export {MyController};

That looks good, right? Well....

When MyController.addOrder() gets called, with this code, the get() method is called on the service, and... BOOM! Error. It says there is no _updateOrders() on this. What? What happened?

Well, it's not on your scope. Why? Because ES2015 has changed how scope is handled, especially within the context of a class. "this" is not the same in the context of the Promise's then(), at this point. But then, how are you supposed to reference other methods of your class?

Bom! Bom! BAAAAHHHHH! Use an arrow function. "Wait? What?" (lot's of confusion today) That's right, an arrow function. From MDN:

An arrow function expression (also known as fat arrow function) has a shorter syntax compared to function expressions and lexically bind the this value (does not bind its own this, arguments, super, or new.target). Arrow functions are always anonymous.

If you aren't still confused at this point you are a rockstar. Basically what it says is that this will become of the context from which you're using the arrow function. So, in terms of a Promise, if we change our addOrder() method like this:

view plain print about
1addOrder (order) {
2        // ... do some stuff
3        // get original
4        this._dataService.get(order.id)
5            .then((order) =>
this._updateOrders(order))
6            .catch(function (error) {
7                // do something with the error
8            });
9    }

This then fixes our this scoping problem within our then. Now, I know this isn't much in the way of an explanation into "How" it fixes it (other than setting the right this), and I know I'm not explaining what an arrow function is either. Hopefully this is enough to stop you from banging your head against the wall anymore, provides a solution, and gives you some clues on what to search for in additional information.

So, as always I welcome your feedback/suggestions/criticisms. Feel free to add a comment below, or drop me a direct line through the "contact" links on this page.

Death to Var - Why Let and Const Really Interest Me In JavaScript

Today I want to talk about the value of ES2015's new let and const variable declarations, and give you some use case scenarios. But first, let me tell you why I was really looking at all.

Ben Nadel is one of my favorite people. You will not, ever, meet a nicer guy. Ben is the kind of guy that if the two of you were walking down the street in a blizzard, and you were cold, he'd give you the shirt off of his own back and go topless so you wouldn't freeze. Yes, he really is that nice of a guy.

I'd like to say that I've learned many things from Ben over the years. He blogs about everything as he learns it, sharing what he finds along the way. And he's the first to tell you that he's not always right. Sometimes the comments to his posts are even more informative than the posts themselves. And, sometimes, he gives his opinion on a matter of programming and that opinion might not always follow best practice.

About a week ago, Ben posted an article titled Var For Life - Why Let And Const Don't Interest Me In JavaScript. He's very clear, in his post, saying that his article is an opinion piece. His thoughts are clear, his examples make sense, and it's easy to see where he's coming from. You'll also find some really thought provoking discussion in the comment thread both for and against.

But I think it's important to truly explore these new constructs in JavaScript. They were introduced with one true goal in mind: to help manage memory in our applications. With the proliferation of JavaScript based applications, both client-side and server-side, the need to carefully analyze our architecture has increased a dozen fold. How you manage your variable declarations will directly impact your overall memory utilization, as well as assist you in preventing race conditions within your app. The let and const declarations really fine tune that control.

The let declaration construct is fairly straightforward. It is a block level scoping mechanism, supplanting var usage in most situations, and controls the "this" level access of those variables. The var declaration construct was a function level scope. What's the difference between block level scoping and function level scoping? Consider the following:

view plain print about
1for (var i = 0; i < 10; i++) {
2        console.log('i = ', i);
3    }
4    console.log('now we are outside of our block. i = ', i); // i now equals 10

Function level scoping means that variables declared using the var construct are available only within the confines of that function, but are not restricted to the block they are declared within. Running the above example shows you that i still exists outside of the for loop block. What happens though if we change that declaration to a block level declaration?

view plain print about
1for (let i = 0; i < 10; i++) {
2        console.log('i = ', i);
3    }
4    console.log('now we are outside of our block. i = ', i); // throws an error that i doesn't exist

In the case above, the variable i is now a block scoped variable and, as such, is only available within the confines of the for loop. The variable is cleared from memory once execution is complete (since there are no references created to those variables in the block), and their values are not available outside of the block, reducing the opportunity for race conditions.

Probably the most misunderstood of these constructs is the const form of variable declaration. Most still think of this as setting an immutable constant, but that's not entirely correct. Let me give you an example:

view plain print about
1const myVar = 'JavaScript is really ECMAScript';
2    console.log(myVar.replace('really ', ''));
3    myVar = 'Purple Haze'; // This throws an error, because you can't do this

OK, that example supports that whole "immutable constant" kinda thing. But that isn't the whole story. Let's look at another example:

view plain print about
1const myVar = {};
2    myVar.foo = 'bar';
3    console.log(myVar.foo);
4    myVar = {}; // You were just fine til you got to this line

"Wait? What?" Yes, you can change a variable declared with const. Sorta.

When you set a variable with const, you are assigning a variable to a specific location in memory. It is set to the type you initially assign. You can adjust properties of that variable, but you can not replace the variable, even with one of the same type. This is why examples with a simple type (string or numeric or boolean) would throw an error, but you could create and remove and adjust object keys or array elements all day long. The variable itself isn't constant, it's location in memory is.

Which allows me to change an example from a previous post. In that post, I talked about using implicit ES2015 getters and setters, and showed an example of broadcasting a variable change in a service from within a custom setter method. I had a variable in my Controller that was not passed in to the Service by reference, so any time I changed the Service variable it had to broadcast that change to my Controller so I could update the controller level variable. In my original example, the variable was originally assigned to the class' "this" scope. But with const I can assign that variable and hold it's location in memory, thereby passing the memory reference and changing how I can control workflow.

view plain print about
1'use strict';
2    
3    class MyController {
4        constructor ($scope, dataService, orderService) {
5            this.$scope = $scope;
6            this._dataSvc = dataService;
7            this._orderSvc = orderService;
8            
9            const myCrazyVar = {};
10            // setting to 'this' too, for controller public accessable reference
11            dataService.myCrazyVar = this.myCrazyVar = myCrazyVar;
12        }
13    }
14    
15    myController.$inject = ['$scope', 'dataService', 'orderService'];
16    
17    export {MyController};

view plain print about
1'use strict';
2    
3    class DataService {
4        constructor () {
5            this.myCrazyVar = null;
6        }
7    }
8    
9    export {DataService};

view plain print about
1'use strict';
2    
3    class OrderService {
4        constructor (dataService) {
5            this._dataSvc = dataService;
6        }
7        
8        add (order) {
9            // update our shared data
10            this._dataSvc.myCrazyVar.orderid = order.id;
11        }
12    }
13    
14    orderService.$inject = ['dataService'];
15    
16    export {OrderService};

Is this wise? I'm sure if you aren't careful you can create issues. But, by passing that memory reference around you also eliminate the need to duplicate variables and broadcast events unnecessarily, reducing your memory footprint and cpu utilization.

Learning when to use let and when to use const will take some time for many who've worked with JavaScript for any length of time. I'm sure this will be one of those new features that takes some significant time to gain true traction among developers. In the end run, it will force us all to think ahead about the architecture of our applications in advance (always a good thing), and the impact of our code on performance.

Now, if I can just convince Ben ;)

AngularUI Router and it's trasitionTo() Method

Angular applications are, typically, single page applications. So when you move around in an Angular application and an area (or areas) of the screen changes, you're said to be moving from one "state" to another. To simplify handling these states you can utilize the very powerful Angular-UI Router. From the GitHub site:

AngularUI Router is a routing framework for AngularJS, which allows you to organize the parts of your interface into a state machine. Unlike the $route service in the Angular ngRoute module, which is organized around URL routes, UI-Router is organized around states, which may optionally have routes, as well as other behavior, attached.

Yes, Angular's built in ngRoute module can give you the basics. AngularUI Router, on the other hand, provides the developer with a simple way of defining very complex state management. Or it seems simple, until...[WHAM!] you hit the brick wall.

The AngularUI Router is so easy, the documentation is... well, it's "OK". Sometimes you just run into a situation where the documentation is not clear enough. For instance, what if you're trying to call the same state, but with new parameters?

"What do you mean, Cutter?" Take the following example. A user is browsing a category in an online store. On the screen is a layout loaded with inventory of that category. Clicking on an item would bring up the detail of that item. After the detail of that item there are links to additional items, either related to the original, or of a similar nature. Seems pretty straightforward, right?

Wrong! (Well, not wrong, but....) It's not always so straightforward.Categories fall under the app, and inventory falls under a category. This says that we need a nested state approach. To illustrate, let me give you some basic routing here to kick it off.

view plain print about
1$stateProvider
2        .state('store', {
3            url: '/',
4            templateUrl: '/partials/store/index.html'
5        })
6        .state('store.category', {
7            url: '/{cat:\\d+}',
8            templateUrl: '/partials/categories/index.html'
9        })
10        .state('store.category.item', {
11            params: {itemid: null},
12            onEnter: function () {
13                // Do something here
14            }
15        });

The initial state ("store") is the site wrapper (navigation and footer and stuff). The category state would likely include some lookup of inventory for the category, and lay out that inventory on the page. The item state might open a modal for that item, with a title and image. It gets an itemid as a param, but for security reasons it is not in the url. There's also no 'view' associated with the item state, with the onEnter handling any UI changes.

So, what's the problem? Well, the issue is the additional inventory items that display below the item details. If a user is looking at an item, then chances are they're in the "store.category.item" state. The additional inventory links on this view would also link to the "store.category.item" state, with the difference being the itemid being passed. If the param were in the URL it might be different, but since it's intentionally being hidden the URL does not change. So a user clicking on a link with a ui-sref (for the same state they are currently in) just chokes. The same happens if your controller or service tries to call $state.go('same.state', params). You can add a reload option to your $state.go() call, but it will reload all the way up the parent state as well.

The requirement ran as followed:

  • Want to reload a child state, without reloading the parent
  • Want to change the parameters of that state
  • The (to be loaded) child state has no UI of it's own, no template, and no URL params

Initially I just tried something like this:

view plain print about
1<a ui-sref="store.category.item({itemid: item.id})">{{item.name}}</a>

It was really frustrating to find my click wasn't working, and took even longer to figure out it was because I was already in the "store.category.item" state. So then I tried something like this:

view plain print about
1<a ui-sref="store.category.item({itemid: item.id})" ui-sref-opts="{reload: true}">{{item.name}}</a>

But, of course, that would reload the entire state tree, parents on down. This was unacceptable. Finally, I found that I couldn't use the standard ui-sref bits to do what I needed to do. So, I ripped that out, and replaced it with an ngClick method in my controller:

view plain print about
1'use strict';
2    
3    class MyController {
4        // ... constructor and stuff, then...
5        
6        goToItem (itemid) {
7            this.$state.go('store.category.item', {itemid: itemid}, {inherit: true, notify: true});
8        }
9    }
10    
11    MyController.$inject = ['$state']'
12    
13    export {MyController};

view plain print about
1<a ng-click="ctrl.goToItem(item.id)">{{item.name}}</a>

Closer. I could set a break point and see that it was trying, at least. But still not right yet. I scoured the Guide for information, and then went looking through the API Reference, before finally stumbling upon $state.transitionTo().

The $state.transitionTo() method is used by $state.go(), under the hood. But, if you don't read through the documentation, it is easy to miss a little nugget in the description of the reload option:

reload (v0.2.5) - {boolean=false|string=|object=}, If true will force transition even if the state or params have not changed, aka a reload of the same state. It differs from reloadOnSearch because you'd use this when you want to force a reload when everything is the same, including search params. if String, then will reload the state with the name given in reload, and any children. if Object, then a stateObj is expected, will reload the state found in stateObj, and any children.

What??? That's not what the go() method's reload option does? In the go() method, the reload option is strictly a boolean. This says that, if I supply it with a string of the state to reload, that it would reload from that state down through it's children (if it were a parent of the one I was calling). By setting it to the same state I was trying to call, I ensured that it only reloads my item state, inheriting data from parent states. So, I tried it out...

view plain print about
1goToItem (itemid) {
2        let statename = 'store.category.item';
3        this.$state.transitionTo(statename, {itemid: itemid}, {reload: statename, inherit: true, notify: true});
4    }

Success! I could merrily bounce around among my inventory, reloading only the named state as I went. Glorious jubilation all around as the band played on (OK, going a little overboard here). Point is, it worked.

The AngularUI Router is immensely powerful, and can truly provide some complex (yet elegant) state management within your application. I highly recommend reading over the guide to see what might be available to you.

Angular, Data, and ES2015 Classes

I'm really loving the changes introduced in ES2015 (otherwise known as ES6 or the new JavaScript). At work we've transitioned to working in ES2015, and discovering the differences has been both fun and challenging, especially when it comes to changes in how variables are scoped. But we'll save some of that for another day, and talk for a moment about passing data around in an Angular application, and how you can have some fun with ES2015 classes.

In Angular, a Controller can have Services as dependencies. Services are singleton in nature, so a Service shared among multiple components can be used to "share" data, to a degree. First, let's create a simple controller with a few dependencies and a custom variable we want to track both within the controller, and among other bits of the app.

view plain print about
1'use strict';
2    
3    class MyController {
4        constructor ($scope, dataService, orderService) {
5            this.$scope = $scope;
6            this._dataSvc = dataService;
7            this._orderSvc = orderService;
8            
9            this.myCrazyVar = null;
10            dataService.myCrazyVar = this.myCrazyVar;
11        }
12    }
13    
14    myController.$inject = ['$scope', 'dataService', 'orderService'];
15    
16    export {MyController};

Next, let's create some simple method in the orderService that needs access to our custom variable. We can easily do this with the shared dataService:

view plain print about
1'use strict';
2    
3    class OrderService {
4        constructor (dataService) {
5            this._dataSvc = dataService;
6        }
7        
8        add (order) {
9            // get our custom var for quick reference
10            let forNow = this._dataSvc.myCrazyVar;
11            // do a bunch of stuff, then
12            forNow.OrderId = order.id;
13            // give our changes back to our dataService
14            this._dataSvc.myCrazyVar = forNow;
15        }
16    }
17    
18    orderService.$inject = ['dataService'];
19    
20    export {OrderService};

First, some might ask "Cutter, why didn't you just pass that variable into the method?" Well, sometimes you just can't. Others might ask "Isn't it passed by reference?" Scoping changes have adjusted how this works as well. Changing the variable in the service won't automatically update your controller variable. We'll talk more about that in a moment. Still other's might ask "What is 'let'?" That's a conversation about the differences in variable assignment in ES2015, and is really a discussion for another day. Ultimately I used let because I need the variable to be mutable.

But, to explain what I'm doing here, the add() method takes an order. What you can see of the method, it gets our custom variable and applies it to a local variable for easy reference. We update it with data from the order that was passed in. We then reset the Service property with the updated data.

OK, but that syntax with the Service property is odd, coming off of ES5. How does that work? Well, ES2015 classes allow for implicit getters and setters of properties. Consider the following:

view plain print about
1'use strict';
2    
3    class DataService {
4        constructor () {
5            this.myCrazyVar = null;
6        }
7    }
8    
9    export {DataService};

In the past, to access and change a property of an object would have required us to write some kind of getter or setter method. In this example, you can simply access and change the property directly through dot notation. But let's say you want to do something a bit more complex. There may be some bit of pre- or post-process you want to do, either when setting or getting the variable. For this, ES2015 classes allow you to define custom getters and setters:

view plain print about
1'use strict';
2    
3    class DataService {
4        constructor () {
5            this._myCrazyVar = null;
6        }
7        
8        set myCrazyVar (value) {
9            // I can do what I want in here
10            this._myCrazyVar = value;
11        }
12        
13        get myCrazyVar () {
14            // I could do stuff here too, if I wanted
15            return this._myCrazyVar;
16        }
17    }
18    
19    export {DataService};

Where this could come in handy is in that inter-app communication. A Controller can call methods on the Service, to set values and stuff, but the Service can't automatically pass data back to the Controller based on actions from other items accessing the Service (like our orderService interaction above). But, in Angular, we can use event handling and binding in this instance. First, let's put a listener on our controller:

view plain print about
1constructor ($scope, dataService, orderService) {
2        this.$scope = $scope;
3        this._dataSvc = dataService;
4        this._orderSvc = orderService;
5        
6        this.myCrazyVar = null;
7        this._dataSvc.myCrazyVar = this.myCrazyVar;
8        $scope.$on('crazyUpdated', ($event, newValue) =>
this.myCrazyVar = newValue);
9    }

Yes, the arrow function is a different concept for most front-end-only developers. I'm not going in depth on that here, but you can find plenty of info out there about them. The gist here is that if the crazyUpdated event is cast it will pass a new value, that we then use to update the Controller variable. This also tells us that myCrazyVar will always be changed from outside of the Controller. Now let's do some magic to make sure that event gets cast. In our dataService:

view plain print about
1'use strict';
2    
3    class DataService {
4        constructor ($rootScope) {
5            this._myCrazyVar = null;
6        }
7        
8        set myCrazyVar (value) {
9            this._myCrazyVar = value;
10            $rootScope.$broadcast('crazyUpdated', value);
11        }
12        
13        get myCrazyVar () {
14            // I could do stuff here too, if I wanted
15            return this._myCrazyVar;
16        }
17    }
18    
19    DataService.$inject = ['$rootScope'];
20    
21    export {DataService};

So we use our custom property setter. When the value is changed, it automatically broadcasts that change. The Controller then picks up that change, and applies it to it's own internal variable.

So, what have we learned here? Well, first there's some samples on using ES2015 classes as Controllers and Services within Angular. Simple examples, but there you go. Next, we talked about class property getters and setters, both implicit (no need to define, they just work), and explicit. Our explicit example shows where you can apply some additional logic during those processes. This may not be the greatest example, but it shows you that you can do stuff. Probably the greatest usages here will be in data validation, or in splitting concatenated data (like a full name to first and last, for example).

Using those ES6 classes in your app is a matter of importing the classes into your app:

view plain print about
1import {MyController} from './MyController';
2    import {DataService} from './DataService';
3    import {OrderService} from './OrderService';
4    
5    // ... other app init stuff
6    .controller('MyController', MyController)
7    .service('dataService', DataService)
8    .service('orderService', OrderService);

So, it's still all new to me, these changes to scope and classes and the like. But it's fun, and powerful, and has a ton of possibilities. If I screwed something up just let me know. All feedback is welcome.

AngularJS, Jasmine Testing, and Mocking Global Objects

So, you've made the jump to writing tests for your code. You've discovered the value of TDD, and even integrated it into your CI process. Awesome! Well, until your tests fail. Then you bang your head against your desk, trying to figure out how to test "this" scenario, and get totally frustrated by the lack of information. Makes you want to quit writing code and start driving the little cart around the driving range that picks up the golf balls.

We've all been there. This morning in fact. Don't give up, you'll figure it out soon enough. For example, you're trying to test a new method in your code. Within this new method you're referencing an object that's on the global scope. But, of course, you get a 'fail' every time, because your test doesn't know what that object is.

The answer is to mock the object. But how do you mock an object on the global scope? Really, it's easier than you might think. Just decorate your window.

Within your root 'describe' block, you probably have a test global 'beforeEach()' where you're mocking your module. Here, you can create a 'decorator' for Angular's '$window', and attach a mock of your object.

view plain print about
1beforeEach(angular.mock.module('myModule', function ($provide) {
2        $provide.decorator('$window', function ($delegate) {
3            $delegate.foo = {
4                bar: function () {
5                    // just for test
6                }
7            };
8            return $delegate;
9        });
10    }));

Then later, when your tests hit code that references that global 'foo' object, your tests won't bomb. Further, you can attach spies to see that object methods are being called in your test

view plain print about
1it ('myMethod', function () {
2        spyOn(window.foo, 'bar').andReturn(true);
3
4        ctrl.myMethod();
5        $scope.$digest();
6
7        expect(window.foo.bar).toHaveBeenCalled();
8    });

When the test is run, and the 'myMethod()' function is called on your controller, which internally calls 'foo.bar()', the test will see that and resolve it with the 'andReturn()' value you supplied.

This is invaluable when you're writing tests for those areas of your app where you're calling the Google API's, or some script that dynamically embeds a video player or something. Your test isn't worried about their code (that's something else all together). Your test is about knowing that your code does what it needs to do.

As they say, "there's usually more than one way to skin a cat" (I'm really going to have to find out where that coloquial nugget came from), so if you know of another way to perform the same test scenario, please share.

2014 in Review: A Year of Learning

2014 has been an outstanding year for me, in many ways, but perhaps one of the most important things (besides my family) has been continuing to do what I love. I'm passionate about development, and constantly working to learn new things. This is important for any developer, as our world changes so quickly today. New standards, new languages, new frameworks, it's a consistent onslaught of new ideas, and impossible to learn each and every one, but important to get exposure none-the-less.

The early part of the year I was still maintaining a large legacy application. We were in the final stages of migrating some very old pieces of the application into new framework architecture (FW/1) along with a new look (based on Bootstrap 3). When you're working on legacy applications, there are rarely opportunities to dive in to new things, so that project was a nice nudge to explore areas previously untouched. Back in April, though, I took on a new position that had me doing nothing but non-stop new product development. Not only was this a great switch, but the particular tasks I was given had me working with technologies with which I had little or no exposure, and often without a team peer who could mentor me, as many of the technologies were new for the company as well.

Historically, I'm a server-side programmer. But, over the last several years, I've spent a great deal of time honing my client-side skills. I'm no master, by any means, but I've consistently improved my front-end work over that time, and 2014 built upon that considerably as well.

One area I could get some mentoring in was AngularJS. This was a big shift for me, and while I am still learning more and more every day, it has been an exciting change up for me. Angular is extremely powerful and flexible, taking some hard things and making them dead simple (to be fair, it makes some simple things hard too ;) ). Angular is one of those items I wished I had spent more time with a year or so back, because in hind-sight it could have saved me hundreds of hours of work. I'm really looking forward to working more with Angular.

From a software craftsmanship standpoint, I also had to dive in to a slew of technologies to assist in my day-to-day development. I now use Vagrant to spin up my local dev environment, which is a godsend. One quick command line entry, and I'm up and running with a fully configured environment. I went from playing around with NodeJS to working with it day in and day out, writing my own plugins (or tweaking existing ones), and to using (and writing/tweaking) both Grunt and Gulp task runners for various automation and build tasks. To take something as "source" and convert it to "app" with a simple command line entry is the shiznit. How many hours did I waste building my own sprites, and compiling LESS in some app? Now it happens at the touch of a few keys.

Then there are the deep areas that some project might take you. I had to dust off year's old AS3 skills to refactor a Flash based mic recorder. There was some extreme study into cross-browser client-side event handling. Iron.io has a terrific product for queuing up remote service containers for running small, process intensive jobs in concurrency without taxing your application's resources. That lead into studies in Ruby, shell scripting, and Debian package deployment (not in any sort of order), as well as spinning up NodeJS http servers with Express.

Did you know that you could write automated testing of browser behavior by using a headless page renderer like PhantomJS? Load a page, perform some actions, and record your findings, it really is incredibly powerful. It also has some hidden 'issues' to contend with as well, but it's well worth looking into, as the unit testing applications are excellent. Then you might change direction and checkout everything there is to know about Aspect Ratio. It's something you should know more about, when thinking about resizing images or video.

(Did I also mention that I went from Windows to Mac, on my desktop, and Windows to Linux, on my dev server? Best moves I ever made!)

Speaking of video, I got the opportunity to go beyond the basics with ffmpeg's video transcoding. For those unfamiliar with the process, you write a command line entry defining what you want. Basically it's one command with 200+ possible flags in thousands of possible combinations, and no clear documentation, by any one source, on how to get exactly what you want (read: a lot of reading and a lot of trial and error).

If that had been all of it, it would have been a lot, but then I really got to have fun, and got to rewrite a Chrome Extension. Now, I had the advantage that we already had an extension, but I was tasked with heavily expanding on it's capabilities, and it's been a blast. Going from something relatively simple to something much more complex is always a challenge, but doing so when you don't fully grasp the tech at hand is even more challenging. Google has created a brilliant triple tier architecture for interfacing the browser 'chrome' with the pages inside them, and developing advanced extensions with injected page applications has a lot of twists and turns along the way. I've learned enough along the way that I'm considering writing a presention on this process for the upcoming conference season.

So, in retrospect, I went from maintaining a large legacy system to doing cutting edge custom development, learning something new each and every day. Awesome! Now, the downside to this sort of process is that you lose valuable productivity time getting through the learning curve. It's difficult to make hard estimates on tasks that no one else has done before, and measuring success is taken in baby steps. But the upside is that you are constantly engaged, constantly motivated, and those skills will eventually translate into better products down the road. Products that won't incur that same learning curve (not to mention the new ideas that can come from exposure to so many different technologies). I can't claim mastery of any of it, yet, but did gain a solid foundation in most of it, and it just makes me hungry for more.

So, if I have one wish for 2015 for you, dear reader, as well as myself, it is only that you continue to learn every day. Maybe not to the levels described above (for I am a bit of a lunatic), but at least take the chance to branch out into one new area of development in the coming year.

CFQueryReader Update: Support for ExtJS 5 and ColdFusion 11

Due to a donation from Bruce Lomasky (thanks Bruce), I've written a new CFQueryReader for ExtJS 5. CFQueryReader is an Open Source project, available on GitHub, that I began years ago to create a proper reader for parsing Adobe's native ColdFusion query JSON return format into something usable by ExtJS data stores. As ExtJS has changed over the years, I have updated the reader to accommodate new versions. But, I don't do much ExtJS development anymore. It's a great front-end app development framework, but I've changed jobs a few times since co-authoring the first books on the library, and just don't work with the technology on a day to day basis anymore. Bruce had a need, and offered to pay for the development of an updated reader. So I dove back in to the ExtJS pool, to help him fulfill his requirements and put a new reader out there.

ExtJS went through a major rewrite between versions 3 and 4, requiring an entirely new direction. While much is backward compatible, in the move from version 4 to version 5, there were some big changes to ExtJS' data packages. Sencha has always done a good job with creating a rich, dynamic framework for web application development. I have always been impressed with their commitment to improving the product, continuously modernizing the framework, and creating more of a "software craftsmanship" approach to web application development. That said they haven't always done such a great job with ensuring an easy upgrade path for developers. Changes to their data package completely broke some of the use cases of CFQueryReader, requiring some refactoring to accomadate.

And that's OK. Sencha's changes to their data packages are some welcome changes, especially for our little plugin. In all of the past revisions of CFQueryReader, we've extended a base Array Reader class, and written an override method of the core readRecords() method. While this worked, it was really kinda kludgy. What we really needed was a way to transform the incoming data packet prior to the reader really processing the object. With ExtJS, we now have that ability to do just that.

ExtJS 5 introduced a new configuration option for data readers: transform. This new configuration attribute takes a function within which you can manipulate the return data of a store's ajax request prior to the reader actually processing that data. This gives some underlying flexibility that wasn't really there before, especially when using Ext.Direct, but for now you really just need to know the basics.

ColdFusion upper cases object keys when it serializes to JSON. If you are manually creating the structure, there are ways to fix that:

view plain print about
1var myStruct = {
2 "key1": "some value,
3 "Key2": "this one is cased differently",
4 "differentKey3": "this one is more different"
5 };

This is fine, and helps a little bit, but ColdFusion creates an object when it serializes a ColdFusion query object, and you can't control it's casing:

view plain print about
1ColdFusionJson = {
2 COLUMNS: ["KEY1","KEY2","DIFFERENTKEY3"],
3 DATA: [
4 ["some value","this one is cased differently","this one is more different"]
5 ]
6 };

When you build your reader configurations, we no longer worry about this casing, as we will do a case insensitive matching of keys during the pre-process:

view plain print about
1reader: {
2 type: "cfquery",
3 rootProperty: "myQuery",
4 totalProperty: "totalCount"
5 }

NOTE: We will lowercase column names during preprocess, and you should remember that when creating your "dataIndex" attributes of your Model's Field configurations.

CFQueryReader will read ColdFusion query JSON serializations that are:

  1. The root of your return
  2. Nested within a larger return object (struct)
  3. Have been converted with ColdFusion QueryForGrid (not suggested, but supported)

And, because ColdFusion 11 includes the new "struct" type for the queryFormat parameter (even as an argument of your ajax request), CFQueryReader will properly parse that as well.

And, since we can now process the return prior to ExtJS calling it's internal readRecords() method, CFQueryReader no longer extends Ext.data.reader.ArrayReader as it's base class, but the more appropriate Ext.data.reader.JsonReader instead, allowing for a more native access approach under the hood.

I've made the new plugin available, for now, within the Ext5 branch of the repository. If you have the need, take it for a spin, and let me know if there's anything that might require tweaking.

Task Queues with IronWorkers

As most of my readers know, I still love ColdFusion. In 15 years of development there are still very few things that I've found that I can't do quickly and easily with CF. That said, ColdFusion isn't always the right tool for the job. I don't mean that ColdFusion can't do the job, only that it's maybe not the best tool. The UI stuff is one great example. The ColdFusion UI stuff was written so that the most novice of developers can spin something up fast. But, anyone who's had to do something a bit more complex quickly finds the limitations behind it's generated UI code. Another is extremely resource intensive processes. Doing image manipulation on one image occasionally isn't such a big deal, but what happens when you have to process several thousand? As powerful as the language constructs are for manipulating these images, the constant conversion of binaries into Java Image Buffer objects, the increasing back and forth within RAM, it begins to bog your server down to the point of a dead crawl.

This image example is just one of many, and when you're writing enterprise level applications you're going to hit these hurdles, and using ColdFusion in these instances is like using a hammer when you need an Xacto blade. This is when you start looking for better options to these processes, that can interoperate with your current ColdFusion services. In looking for just such an option, I was pointed to Iron.io.

Iron.io is a cloud-based set of services that can be run on many of the major clouds. They began with a distributed Message Queueing service (IronMQ) built for handling critical messaging needs for distributed cloud applications. Building upon their queueing abilities, they also created IronWorkers. IronWorkers are async processing task queues. They allow you to define what your process environment needs to look like, your task script to process, and then you can queue up tasks which can asynchronously run in their own independent container environments. Once queued, IronWorker with run X number of tasks asynchronously (X being dependent on the plan level you choose). Each task runs within it's own sandbox, with it's own independent RAM and processor allocation, so that one running process does not affect another. As tasks complete their sandbox is torn down, the queue continues to spin up the remaining tasks on demand, until the task queue is done.

The ease with which they've developed this service is amazing. Your ".worker" file defines your environment. Each instance is a headles Ubuntu system. You can select from a number of "runtime" setups, allowing you to work in the language that you're most comfortable with (Node, Ruby, PHP, Java, etc), as well as picking a specific "stack" if you want to mix and match the setup a bit. Each instance also already comes preloaded with several common Linux Packages (ImageMagick, cURL, SoX, etc). Within your ".worker" file, any additional files, folders, etc that you require can also be defined, including .deb packages.

Once your worker is defined, assets gathered, scripts written, etc., you then create your worker from a simple command line call. Once the "build" is complete, your new task service is ready to be called. This can be done via command line or (perhaps more commonly) via an http call. You can even define a webhook for your worker that can kick off your tasks from Git or elsewhere. You can pass variables to your task as part of it's "payload". This is just a simple HTTP Post, passing in name/value pairs that can be used within your process script. The "payload" is limited to 64k in size, so any files you may need on the fly (such as images to process) should be retrieved from within your process, most likely from somewhere like S3. Your process does it's thing, your script sends a command to exit the process (process.exit()) and it's done, spinning up the next task in the queue.

There really is a lot more to it. You can get as simple or as complex as you are capable of writing. IronWorkers are extremely powerful, and scale beautifully. After several weeks working on multiple processes, I can also say that their support is exceptional. They have HipChat channels setup to assist people, and they've been extremely responsive and helpful. They also maintain sample repositories of many common tasks, in a variety of languages, to help you get started while working in whichever environment you're most comfortable with.

More Entries