Every single application I've been building since 2 years ago have been a single page application. The server side is a REST API, and the client side is rich and handles the view and web page generation. I've been doing it so much, that I sometimes forget how to build web applications in the old static way. It's my prediction that soon every single website/web application will be using some sort of javascript, and this even includes mostly static content websites, because the advancements in javascript makes possible all sorts of new user interfaces, and interactivity in producing and consuming content on the web.

There is one big problem however. Search engines, social network robots, and friendly scrapers do not execute javascript. This means things like Google Search, Google Adverts, Bing, Facebook, Twitter, LinkedIn and more all don't work or don't work very well with javascript enhanced rich client side applications. The social networks also use bots for embedding content, such as when you share a link.

This has resulted in developers having to choose between a rich interactive application that provides new and exciting user experience, or static reliable webpages. Most businesses end up compromising with static public facing pages, and then internally a rich control panel. Or if they are starting a whole new content based site, they stick with static server side web page generation.

Now why don't the crawlers and robots support javascript execution? Well one thing is that it's difficult. You need a whole javascript runtime, and executing that is not cheap nor can it be done very fast. One of the main issues is that javascript is used to bring in asynchronous content. So the static content on a single page application is just a skeleton, the javascript is executed to fill it up with dynamic content. This means any robot/crawler has to stay on the page for a while waiting for asynchronous content to be loaded. A static HTTP request might be completed in 1 second, but waiting for asynchronous content can triple, or quadruple the time required. So it's a complex and expensive thing to do.

I and Mustafa (a student in Polycademy) set out to solve this problem. After about 9 months of development, let me introduce you to SnapSearch.

SnapSearch is Search Engine Optimisation (SEO) for Javascript, HTML 5 and Single Page Applications. It makes your javascript applications crawlable by search engines and scrapable by social network robots. It works by intercepting requests that is sent to your web application, detects if its coming from a search engine bot or social network bot, and in turn returns a response containing a cached snapshot version of your javascript web page which it acquired by rendering your webpage previously or on the fly with automated Firefox instances.

SnapSearch provides a number of officially supported, professionally developed middleware that can be easily integrated into your current web application stack. It currently provides PHP, Node.js, Ruby and Python middlewares. They are very flexible, and allow you to specify which search engines you want to intercept for, and which you want to ignore.

It's free to get started, there's a 1000 free usages per month cap. You get nice analytics that allows you to figure out which search engines are accessing your site. There's comprehensive documentation for both the API and the middleware. Note that if you're into web scraping and you're looking for a solution to process javascript, you can use SnapSearch for this purpose too, it's very fast, in fact most on the fly renderings happen under 4 seconds, whereas cached snapshots gets returned under 1 second. SnapSearch is load balanced, so we'll scale linearly.

For a limited time offer, use "SNAP" as the code when you sign up and you'll get 2000 free usages per month cap.

The project is under heavy development, and there's still lots more to do. But it's fully functional and production ready. I'm already using it for http://dreamitapp.com/ and https://snapsearch.io/

If you're looking for help setting it all up. Feel free to hit our web chat at Hipchat. I'll be able to answer any questions and help with deployment.

If you're writing a single page application, or web pages enhanced with javascript, you don't need to make any more compromises. Just build it, and know that SEO will always be available to you. Try the demo on your web page, and you can see the difference in the content being served to search engines.

The tech savvy among you might realise this is similar to PhantomJS as a service. But it's not, because it doesn't use PhantomJS. Using real Firefox instances has the advantage of being able to take advantage of Mozilla's 6 week Firefox release cycles. This means we can support the latest in HTML5 technology quicker. The qtwebkit engine that PhantomJS relies on has a slower and sporadic release cycle. And it doesn't support plugins/extensions. SnapSearch has the possibility of adding in plugin/extensions such as Flash (this is under development).

If you're looking to get into single page application development, and you haven't jumped on the bandwagon yet, look no further than AngularJS. It's the best! Also try out Facebook's React as it can easily integrate into AngularJS's directive concept (it can be faster).

The land of client side development is changing very fast, and there is some serious engineering going on. I suggest you to also check out Polymer and Brick. Both of which AngularJS will be taking advantage of when they become more stable. Here's a comparison, you'll see that it's very similar to AngularJS directives.

Polycademy has been improving it's development workflow. Especially the deployment of web server stack.

One of the things that I realised in running the courses, is that the work to setup web server stacks which means the HTTP server, Application Server, Database Server and various other daemons and applications is a very time consuming process. This is mainly devops work and configuration management, it's not really development. What is required is an automatic way to create development and production servers that is isolated from each other, and has the exact same configuration every time you destroy or create one. The development server should match the environment of the production servers so that you reduce the number configuration problems during deployment of your application to the production server. In setting up a workflow like this, you can get a file that specifies how these virtual servers are setup, and just copy paste the file to whichever computer you are currently developing with. In some cases you can just straight up copy the virtual machine image which is the entire operating system and associated installed applications and load this into wherever you want to develop as well.

So I've been spending time researching on deployment automation. And I came across services like Vagrant, Docker, VirtualBox, Chef, Puppet, Ansible, SaltStack and a whole lot more. Enterprises tend to use devops personnel to write up configuration scripts that essentially deploy hundreds or thousands of servers in an automated mannger. This saves time and cost. However these methods are perfectly applicable to the individual developer or startup.

My research has led me to conclude that a combination of VirtualBox, Vagrant, Docker, and Ansible is the best development and deployment workflow for individual or startup projects. VirtualBox is for setting up virtual operating systems on your computer, this is only for development, in production your virtual operating systems are provided to you when you rent a VPS from a hoster such as Digital Ocean. Vagrant is a command line tool that allows you auto configure VirtualBox. Docker is for setting up lightweight virtual machines inside the virtual operating system, and these will run the applications you're developing. It's like a virtual machine inside a virtual machine. I don't yet understand the specifics of how Docker works, but when I do I'll probably write a blog post about setting up your own private cloud host. Finally Ansible the configuration management/IT automation tool, that will basically setup Docker and other services inside the Virtual Machines, and perhaps setup your application inside Docker as well. Note that another service called Packer allows you to quickly replicate your virtual machine images so it's good for the development of virtual appliances.

So in order to learn how to do all of this, I had start at the beginning with VirtualBox. I've managed to setup VirtualBox as a guest operating system that not only has a synchronised development folder with my host operating system (Windows, Linux and Mac) but is also accessible over the network.

What does this all mean? This means I can open up my Sublime Text Editor to my "www" directory. This is where all my projects reside. I can edit a file in this directory. This file change is automatically propagated to the virtual machine instance. Then in my host machine I can open up Firefox to "http://localhost:8000" or "http://dev.test.com:8000" and get to see my updated change in my application. Then in my host or guest machine, can I commit and synchronise to github. All of this can be done without installing a web server stack (HTTP servers, Application Runtimes..etc) on my host machine, thus keeping my host machine clean. I'm not talking about production deployment at this moment, because I haven't got that far with Docker..etc. Therefore any deployment routines to production servers will still stay the same. However I'm still quite happy with what I have here, it is definitely an improvement over running WAMP or MAMP. As I can easily use NGINX and other Unix based tools. Furthermore, the instructions below can easily be adapted to setting up SSH access from your host machine too if you want to command your guest operating system from your host's terminal.

Now comes the tutorial of how to set this all up. This part does require you understand Ubuntu Linux, basic bash scripting, CLI usage, modifying the hosts file and setting up NGINX.

First download VirtualBox: https://www.virtualbox.org/wiki/Downloads. Then download Ubuntu Desktop: http://www.ubuntu.com/download/desktop

Run VirtualBox and create a new virtual machine instance. A wizard dialog will pop up. Make sure to select the correct settings for your virtual machine. The perfomance of the virtual machine will be better if you give it a static size hard drive and 2048 mb ram, 2 CPUs and 32 mb of Video Memory along with 3D acceleration. These settings can be changed later inside the GUI of VirtualBox, they make your VM more expensive to use, but it also makes your VM a lot more faster and responsive. Once your virtual machine instance is set, you need to start it up. Then load the ISO of Ubuntu image you just downloaded. Follow the instructions to install Ubuntu in your new virtual operating system.

Once it's all been installed, you need to remove the ISO CD from your virtual operating system. This can be done by shutting down your VM (virtual machine), then going into the settings of your VM inside VirtualBox and removing the CD in the storage settings. Start it up again and this time install the VBoxAdditions ISO. The VM would have prompted you to do this. This VBoxAdditions allows your VM to setup shared folders and other good things. We'll need shared folders to allow your host machine to edit files and have your guest machine use those updated files. Once that is done, you'll need to restart your VM and remove the VBoxAdditions ISO CD.

Setting up shared folders is quite easy. Shutdown your VM. Go into the VirtualBox settings for the VM. Go into shared folders. Add a new shared folder record. The first folder path is the path your development folder. For me this is "D:www". The folder name would be "www". Make sure Read-only is off, switch on Auto-mount and Make Permanent. Hit Ok. Start your VM. Using your terminal in root user assuming that your folder is "www", run:

cd /media/sf_www/
ls -l

This should show you the contents of your "www" folder on your host machine. If your folder was called something else, it will always have the prefix of "sf_". Furthermore we can see the owner and group the files and folders are associated with. This folder will be grouped under "vboxsf". Right now we can only access it with the root user. In order to add extra users to this group we can use this bash command (fill "username" with the name of the user):

sudo adduser username vboxsf

We should add the default user that you log in with, and also the "www-data" user. The "www-data" user will be the user that your web server stack will use. Your webserver such as NGINX, PHP, Cron scripts should not be using root or your default user, but a custom user for web stuff. It doesn't need to be called "www-data" but that is the default name used by PHP-FPM.

Once you do that, the setting does not take effect immediately. You need to restart your VM. After which you no longer have to be a root user to access the "sf_www" directory inside your VM.

Now the /media/sf_www is not really convenient directory path. So let's try to access it from something like /www. To do so, setup a symbolic link from /media/sf_www to /www.

ln -s /media/sf_www /www

Now test it out. Go into the "www" directory from your host machine, and create or edit a file. Now in your VM, check the /www directory and see if the file is there or edited. We should now have synchronised development folders!

Now let's do some port forwarding. First we need an HTTP server on the VM. So we can install NGINX which by default listens on port 80 and shows a default NGINX page.

sudo apt-get install nginx

Test it out by launching Firefox inside the VM and heading to "http://localhost". If it shows the page, then NGINX is working. While you're at it, try out Polycademy's NGINX configuration best practices: https://github.com/Polycademy/WebserverConfiguration

Shutdown the VM. Inside VirtualBox's VM settings, go into Network. From Network, the Network Adapter should be set to NAT. Click on Port Forwarding. Add a new record. In this record, set the Host Port to 8000 and the Guest Port to 80. The Protocol should be TCP. The name and everything else can be left as default. You can change the name to anything you like. Restart the VM.

Now in your host machine, launch your browser and head to "http://localhost:8000". This should show the same page as the page you saw when you visited with Firefox inside the VM. In the future you can also port forward HTTPS connections, and you'll need to go from a random host port such as 8001 to the port 443 on the guest machine, this would result in URLs like https://localhost:8001.

You can change the hosts file inside the host machine independently of the guest machine. This means the hosts file inside the guest machine affects the guest. So if you want a development URL such as "http://dev.example.com" inside the guest machine, just set that up inside /etc/hosts in the guest machine. This will work for browsers inside the guest machine. But if you want that for the host machine, you'll need to add that record to the hosts file in the host machine.

Now you can install all your web server stack into your VM and not worry about the host machine or incompatibilities between operating systems especially when you develop on Windows/Mac but deploy on Linux.

If you meet any problems, check the NGINX logs which should be specified inside the "nginx.conf" file inside /etc/nginx/nginx.conf. There could be issues with file caching with NGINX, which means that NGINX is caching the files it's serving. This is not good for development, and should be switched off. You should then restart NGINX with sudo service nginx restart. Other problems could include permissions like "vboxsf" but this should work great.

The next step would be learning Vagrant & Ansible to automate the VM constructions as it builds on top VirtualBox (and other various VM provisioiners). Currently all you have is a VM image you could share, but sharing a 1 gig file is a bit hard, a configuration file would be smaller and easier to work with.

I've recently began on a small client project involving a Wordpress site for Auto Timor Leste - Toyota East Timor. I don't usually work on Wordpress, but the last time I worked with it, I realised it was a clunky blogging platform that is quite difficult to work with when you want to do more than just blogging. Nowadays there are better alternatives like Docpad, Octopress and Jekyll but those are still in the early adopters stage with developers. So Wordpress does have the advantage of being widely known and easily supported by developers. It also has a whole set of plugins. So you can say it's made for the non-developer. With that in mind, I suggested to use Wordpress as Toyota's official website framework.

When I was working Wordpress a while ago, the deployment process involved downloading the official Wordpress installation via zip file. Unzipping it into your web root directory, running the initial configuration, hardcoding any configuration required in the wp-config.php file and then finally FTPing into an online server usually purchased from Lunarpages.

Such a workflow has several disadvantages. The worst is FTP. FTP is very slow. It transfers each file individually one by one, and each time it does this, it needs to reestablish the connection. This can be a real hassle when you have thousands of files, and the Wordpress 3.7.1 framework has as of now 1176 files. And if you hit any interruption in the connection, you may need to restart the whole process since you don't know which file was half uploaded and corrupted. The other problem is that server differences will require reconfiguration on the remote server. This generally means saving the config file separately, modifying it to match the necessary remote server configuration which could involve database settings and database migration, uploading it via FTP to the remote server, access the site from your web browser and debug things manually. Oh and you have do this all over again if you need to install plugins or fiddle with the theme. This leads us to our second problem: dependency management and version control.

Storing the wordpress project as files on your work computer makes it difficult to work flexibly. What if your computer dies? What if you want to work on a different computer? What if you need to share code with a team and each needs to contribute in different ways to the overall project? So you might decide to copy paste your working folder in a backup harddisk, and copy it to a USB thumbdrive so you can work on the go. Now you have the problem of version control. How do you reconcile different versions of the project elegantly. If you changed one file on the USB thumbdrive, then this becomes the new master version, which has to be manually copied to all the other locations where you stored the code. You might meet merge conflicts, where 2 files might have been updated to different code at the same time, which version is the right one to use, and what if both versions solve different problems. And this problem becomes compounded if you're working in a team. Furthermore, this constant copying and pasting results in a lot of duplication, especially if the majority of the project are external dependencies, that is plugins or themes or the Wordpress framework itself, which is code written by people outside of your current scope. There should be some way of isolating your application's code from dependencies so that there's less duplication of the source code, and you only need to bring in the dependency when it's actually being executed.

Now I am digressing, of course there has been solutions to this distributed source control via SVN and friends for large software projects, but if you're a single developer, there are still many lessons to learn from large software development workflow.

So I began looking a different, more modern way. Nowadays we have PAAS (platform as a service) businesses that offer a more streamlined way of deploying web applications. They usually operate via a customised deployment routine similar to Capistrano, Dokku, Git based deployment, or some form proprietary continuous integration. All of these systems provide deployment hooks, which is essentially automated code that is ran after the code has be transported to the remote server. This solves the FTP problem because they package up the directory you want to send into a compressed archive, and once it's uploaded, it's extracted to the web root location. This speeds up the deployment process immensely, and it's more secure as well.

But there is still the problem of configuration and dependency management. So I researched a bit and found articles regarding Wordpress and Composer and Git. Here are my sources: Using Composer with Wordpress and Deploying Wordpress using Git and Capistrano. You should read those sources before going further.

Those sites are really good sources, but they left a bit of detail out. So I'm going to fill in some gaps:

Start by getting Composer (dependency management tool) to install Wordpress as a dependency. Now Wordpress has not been registered as a package on Packagist (Composer's department store). But this is not a problem. We can use a custom installer which will bring in Wordpress as a zip file and unload it into the project directory. Here is the composer.json

{
	"repositories": [
		{
			"type": "package",
			"package": {
				"name": "wordpress",
				"type": "webroot",
				"version": "3.7.1",
				"dist": {
					"type": "zip",
					"url": "https://github.com/WordPress/WordPress/archive/3.7.1.zip"
				},
				"require": {
					"fancyguy/webroot-installer": "1.0.0"
				}
			}
		}
	],
	"require":{
		"php": ">=5.3.0",
		"wordpress": "3.7.1"
	},
	"config": {
		"bin-dir": "bin/",
		"process-timeout": "1000"
	},
	"extra": {
		"webroot-dir": "wp",
		"webroot-package": "wordpress"
	}
}

When running composer install, this will download https://github.com/WordPress/WordPress/archive/3.7.1.zip and utilise this "fancyguy/webroot-installer": "1.0.0" custom installer to extract the archive into your root directory. The "extra": { "webroot-dir": "wp", "webroot-package": "wordpress" } becomes important. The "webroot-dir" specifies what the wordpress directory's name, while the webroot-package specifies what package we want the custom installer to use. To get different versions of Wordpress, just change the "3.7.1" version to desired Wordpress version. There are three locations to change, the "version": "3.7.1", "url": "https://github.com/WordPress/WordPress/archive/3.7.1.zip" and "wordpress": "3.7.1". The Wordpress framework is downloaded from the Github mirror. Because Wordpress is a rather large dependency, I set the "process-timeout": "1000" to 1000 seconds, since a slow internet connection may cause your installation to timeout. Composer uses 300s by default. Your directory layout should look like this:

vendor
wp
composer.json
composer.lock

Now let's configure Wordpress so it understands that it's in its own directory. Follow the Using a pre-existing subdirectory install instructions here Giving Wordpress Its Own Directory. The General panel means going to http://pathtoprojectdir/wp/wp-admin, then going to Settings > General and change the Site Address Url but not the Wordpress Address Url. When you first get there, it'll ask you about the wp-config.php file, just say yes. Make sure you've got the database settings setup already, this is just like a normal Wordpress installation. After you've copied the "index.php" (note that the .htaccess file probably does not exist anymore) to your project root, you need to make the index.php's require command point to the Wordpress directory. Which going to be like this snippet:

/** Loads the WordPress Environment and Template */
require( dirname( __FILE__ ) . '/wp/wp-blog-header.php' );

You should now copy the wp-config.php or wp-config-sample.php into your project root and rename it to wp-config.php. Next copy the whole wp-content directory into your project root as well. Your project directory should look like this:

vendor
wp
wp-content
composer.json
composer.lock
index.php
wp-config.php

Now we have to configure wp-config.php so that Wordpress understands that our application code is in wp-content and the wordpress installation is in wp. Here's what my wp-config.php file looks like:

<?php

define('ENVIRONMENT', 'development');

/**
 * Automatic Url + Content Dir/Url Detection for Wordpress
 */
$document_root = rtrim(str_replace(array('/', '\'), '/', $_SERVER['DOCUMENT_ROOT']), '/');

$root_dir = str_replace(array('/', '\'), '/', __DIR__);
$wp_dir = str_replace(array('/', '\'), '/', __DIR__ . '/wp');
$wp_content_dir = str_replace(array('/', '\'), '/', __DIR__ . '/wp-content');

$root_url = substr_replace($root_dir, '', stripos($root_dir, $document_root), strlen($document_root));
$wp_url = substr_replace($wp_dir, '', stripos($wp_dir, $document_root), strlen($document_root));
$wp_content_url = substr_replace($wp_content_dir, '', stripos($wp_content_dir, $document_root), strlen($document_root));

$scheme = (isset($_SERVER['HTTPS']) AND $_SERVER['HTTPS'] != 'off' AND !$_SERVER['HTTPS']) ? 'https://' : 'http://';
$host = rtrim($_SERVER['SERVER_NAME'], '/');
$port = (isset($_SERVER['SERVER_PORT']) AND $_SERVER['SERVER_PORT'] != '80' AND $_SERVER['SERVER_PORT'] != '443') ? ':' . $_SERVER['SERVER_PORT'] : '';

$root_url = $scheme . $host . $port . $root_url;
$wp_url = $scheme . $host . $port . $wp_url;
$wp_content_url = $scheme . $host . $port . $wp_content_url;

define('WP_HOME', $root_url); //url to index.php
define('WP_SITEURL', $wp_url); //url to wordpress installation
define('WP_CONTENT_DIR', $wp_content_dir); //wp-content dir
define('WP_CONTENT_URL', $wp_content_url); //wp-content url

/**
 * Secrets
 */
require_once('Secrets.php');
Secrets::load();

/**
 * MySQL settings
 */
if(ENVIRONMENT == 'production'){

	define('DB_NAME', $_ENV['secrets']['database_name']);
	define('DB_USER', $_ENV['secrets']['database_user']);
	define('DB_PASSWORD', $_ENV['secrets']['database_pass']);
	define('DB_HOST', $_ENV['secrets']['database_host']);
	define('DB_CHARSET', 'utf8');
	define('DB_COLLATE', '');

}else{

	define('DB_NAME', 'autotimorleste');
	define('DB_USER', 'root');
	define('DB_PASSWORD', '');
	define('DB_HOST', 'localhost');
	define('DB_CHARSET', 'utf8');
	define('DB_COLLATE', '');

}

/**#@+
 * Authentication Unique Keys and Salts.
 *
 * Change these to different unique phrases!
 * You can generate these using the {@link https://api.wordpress.org/secret-key/1.1/salt/ WordPress.org secret-key service}
 * You can change these at any point in time to invalidate all existing cookies. This will force all users to have to log in again.
 *
 * @since 2.6.0
 */
define('AUTH_KEY',         $_ENV['secrets']['auth_key']);
define('SECURE_AUTH_KEY',  $_ENV['secrets']['secure_auth_key']);
define('LOGGED_IN_KEY',    $_ENV['secrets']['logged_in_key']);
define('NONCE_KEY',        $_ENV['secrets']['nonce_key']);
define('AUTH_SALT',        $_ENV['secrets']['auth_salt']);
define('SECURE_AUTH_SALT', $_ENV['secrets']['secure_auth_salt']);
define('LOGGED_IN_SALT',   $_ENV['secrets']['logged_in_salt']);
define('NONCE_SALT',       $_ENV['secrets']['nonce_salt']);

/**#@-*/

/**
 * WordPress Database Table prefix.
 *
 * You can have multiple installations in one database if you give each a unique
 * prefix. Only numbers, letters, and underscores please!
 */
$table_prefix  = 'wp_';

/**
 * WordPress Localized Language, defaults to English.
 *
 * Change this to localize WordPress. A corresponding MO file for the chosen
 * language must be installed to wp-content/languages. For example, install
 * de_DE.mo to wp-content/languages and set WPLANG to 'de_DE' to enable German
 * language support.
 */
define('WPLANG', '');

/**
 * For developers: WordPress debugging mode.
 *
 * Change this to true to enable the display of notices during development.
 * It is strongly recommended that plugin and theme developers use WP_DEBUG
 * in their development environments.
 */
define('WP_DEBUG', false);

/* That's all, stop editing! Happy blogging. */

/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') )
	define('ABSPATH', __DIR__ . '/wp/');

/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');

It probably looks quite different from yours. There's a quite a bit of important logic working here. Let's go through it. The first part defines the ENVIRONMENT constant. This will be useful for our branching off our database configuration. The next we do some magic URL detection. Remember the part in the General panel where you had to set the URL to the Wordpress installation and the site URL. It's probably along the lines of "http://localhost/pathtoyoursite". This will be a bit of a problem if you upload it to a remote server, and the URL will probably be your production URL having a real domain. We don't want to do so much configuration work, so I wrote some code to automatically detect the URL to the project root and other relevant areas such as:

define('WP_HOME', $root_url); //url to index.php
define('WP_SITEURL', $wp_url); //url to wordpress installation
define('WP_CONTENT_DIR', $wp_content_dir); //wp-content dir
define('WP_CONTENT_URL', $wp_content_url); //wp-content url

This allows our Wordpress project to work with any URL on any server, whether it's your development server or production server. I saved this portion of the code as a gist:

The next portion loads the Secrets class and loads secrets. What are these secrets? Well they are the production database configuration and authentication keys. These are required since it is not a good practice to save production configuration keys to version control. I will get back to this and the database/auth configuration in a moment. At the bottom, we made sure that the require command required from the "wp" directory.

/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') )
	define('ABSPATH', __DIR__ . '/wp/');

/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');

Let's get the Secrets working. First we need a secrets directory which will hold keys.php. Something like this:

<?php

//encryption

$secrets['auth_key'] = '';
$secrets['secure_auth_key'] = '';
$secrets['logged_in_key'] = '';
$secrets['nonce_key'] = '';
$secrets['auth_salt'] = '';
$secrets['secure_auth_salt'] = '';
$secrets['logged_in_salt'] = '';
$secrets['nonce_salt'] = '';

//production database details

$secrets['database_host'] = '';
$secrets['database_name'] = '';
$secrets['database_user'] = '';
$secrets['database_pass'] = '';

The above will be inside "secrets/keys.php". This file will be ignored in version control. This is a good practice. Next we'll get the Secrets class which I wrote to facilitate the loading of secrets. It will include any php file inside the secrets folder, and pass all properties that is part of the $secrets[] array into the $_ENV['secrets'][] array. Here is a gist of the Secrets class:

The Secrets.php should be committed to version control and should be at the project root. You project structure should look like:

vendor
secrets
wp
wp-content
composer.json
composer.lock
index.php
wp-config.php
Secrets.php

Now you can see how the secrets can be utilised in production settings. On the production side, you would write a deploy hook that downloads your keys.php into the secrets folder which would be kept in a separate location that is of course secret. You can use Dropbox, private git then curl.

Now that we have separated the framework from our application code, which mostly kept in the wp-content directory, because that's where our themes and plugins go, we should go a step further and use Composer to manage any plugin dependencies. Now most Wordpress plugins are not Composer compatible. But the guys at http://wpackagist.org/ as mirrored every single Wordpress plugin as a composer installable package. All of which use a custom installer to install into the "wp-content/plugins" directory. I installed the WP-Migrate-DB plugin via Composer. You can do this by adding their location as a repository location as seen in this snippet:

	"repositories": [
		{
			"type": "composer",
			"url": "http://wpackagist.org"
		},
		{
			"type": "package",
			"package": {
				"name": "wordpress",
				"type": "webroot",
				"version": "3.7.1",
				"dist": {
					"type": "zip",
					"url": "https://github.com/WordPress/WordPress/archive/3.7.1.zip"
				},
				"require": {
					"fancyguy/webroot-installer": "1.0.0"
				}
			}
		}
	],

Then you can install the plugins in the require position:

	"require":{
		"php": ">=5.3.0",
		"wordpress": "3.7.1",
		"wpackagist/wp-migrate-db": "0.5"
	},

Now how do we find out the exact name and version of the Wordpress plugin that we want? Well you need to go into the plugin SVN directory: http://plugins.svn.wordpress.org/ and do a Ctrl+F to the plugin name, and that's the correct name to use. Click on the plugin, and you'll find the version you will want. Run composer update and this will install the plugin into "wp-content/plugins/pluginname/".

Let's now commit this to Git version control. However before we do so, we need to add some directories to the .gitignore. These ones to be specific:

#################
## Custom
#################

vendor/*
bin/*
secrets/*
!secrets/*.example
!secrets/*.gitkeep
wp/*
wp-content/backup/*
wp-content/cache/*
wp-content/plugins/*
wp-content/upgrade/*
wp-content/uploads/*

We have to ignore those files and folders because they don't belong in the version control. The secrets are self-explanatory. The wp folder is a dependency which can be brought in via Composer, there's no need to duplicate third party source code in our source control. The wp-content folders are ignored for the same reason. The uploads and backup and cache directory are actually meant to be managed outside the source control because they do not constitute the source code of the software application. Of course when you want to migrate to another server, creating a server snapshot or a database dump. For best case practices, your file uploads should go to a completely separate file sever such as Amazon S3, so the storage, backup and serving of the files can be abstracted from the logical component which is your software application source code.

So this blog post was a bit longer than I thought it would take, and it is more catered to those who already understand bits of Wordpress and how Composer works. But I hope it helps you to make your Wordpress workflow better.

I want to start blogging more about my technical journeys through Polycademy. While I was building Dream it App (http://dreamitapp.com), I noticed that there would FOUCs (flash of unstyled content) in between pages on the same application. Dream it App is a single page application, so state transitions are meant to be more fluid and closer to a desktop experience. In a traditional static page, you get FOUCs/FONCs (flash of no content) all the time between changing pages or changing sites, usually it's a white background. You'll notice this when you're on Google and you navigate to a particular new site you have been to before. But in way SPAs should be able to avoid this problem. Unfortunately for most cases, you trade the page transition white background FONC for a FOUC that has the template and styles loaded, but the data/content not loaded. This is happening because SPAs are asynchronously loading resources, so things don't come in one at a time. But then I saw Youtube's new loading mechanism.

If you've been to Youtube recently, you may have noticed a loading mechanism. Go any Youtube video, and then click on one of the related videos without opening a new tab or window. You should see this:

It looks like a loading bar. But this is not what this post will be talking about. A loading bar is fairly easy to implement in AngularJS, see the ngProgress module. What Youtube is doing is preloading all the data and content of the next video before it transitions the view state to that new video page.

So I began thinking overnight on how to achieve this. This is a difficult thing to implement. When writing an AngularJS application, you have controllers that associate themselves to various view states. A basic example would be that every single page (delineated by the URL) has a controller. Then you put all the commands that load data asynchronously from the server, and all the modifications of data into the controller. You might even abstract the process and modularise but in the end it's all activated from the controller. In most use cases, only when the page has transitioned to the new page will the controller's code run and hence bring in all of the content for the new page. So there has to be some way to load data asynchronously before activating the controller. One couldn't just run the controller before changing page, because the controller may be relying on the DOM as a dependency to run other things.

The first obvious solution would the preload resources upon the first load. In some sites, they preload images and cache them in the browser before you even load a page that needs those images. This could work as well for a small site that has fairly static content. However when you get to scaling up dynamic content it doesn't work. It does work well for templates. In fact Polycademy's current Angular architecture is to preload all templates and cache them upon the first page load. This is because templates are usually quite small when they are all combined. But this doesn't solve the problem since the content is still being loaded asynchronously and being added to the template. So when you transition state, you might see the template, but no data. What we need is a method to separate the activation of loading asynchronous content that is on demand and relevant to the specific page from the controller's code. So that upon running the controller's code (and transitioning the view), the data is there already. But this all needs to be done JIT (just in time) for the page, not for all pages.

So I did some googling. And there is a method to do this. It's called the resolve property on the routes. It is possible to configure AngularJS's routes to each have a resolve property, which calls functions that return a promise, which all have to be resolved, before finally dependency injecting them into the controller.

The methods are listed here: Delaying AngularJS route change until model loaded to prevent flicker and AngularJS - Behave like Gmail using Routes with Resolve and AngularJS Video Tutorial: Route Life Cycle

I haven't got around to testing those methods in production. But I hope it also works with the back button. Youtube does its preloading mechanism on the back button as well. At any case, this is a boon to user experience. One of the major factors of using SPAs to is to have a more responsive experience, and seeing FOUCs as you transition between views deteriorates that experience. You would never see a well designed desktop app with FOUCs in between transitions.

I also use ui-router for a more sophisticated state machine routing. And ui-router also supports resolve. So I'll be implementing this in Polycademy's next AngularJS projects.

Also for images to be resolved prior to the state change, you can use AngularJS to HTTP preload them. See this Stackoverflow post: http://stackoverflow.com/questions/18627609/angularjs-resolve-to-wait-until-external-images-have-loaded This form of eager loading for each state change is required when the layout of the next page depends on the image's size which may not be determined until is loaded. This obviously refers to dynamic or user uploaded images, not image assets of your site, which you do know what the size is beforehand.

It has been a while since our last blog post, but we've had quite a bit of activity here for the last several months.

When we finished term 1, our students created various MVP versions of their ideas. Two of them can be seen online: Buhloon and ChessDimensions. The students found out that learning software development is an arduous process. But it's something that can be learned with commitment, and also something that allows you to create original applications in a short amount of time. It's also a life long learning process, since they've only scratched the surface of application development.

After and during term 1, we learned new development techniques. We wanted to streamline some of the course content, since during term 1 we transitioned between static web applications to more dynamic interactive applications, facilitated by single page frameworks such as AngularJS.

Term 2 unfortunately didn't run, since I was finishing up University examinations as I was about to graduate from the Australian National University. Once I had graduated, I decided to move Polycademy to Sydney.

So Polycademy is now located at the Vibewire co working place. It's in the heart of Sydney, close to Fishburners and Central station. This place is designed for social enterprises and organisations, so we're in good company.

Just before I launch into talking about what's happening in term 3, I want to point that we finished building an application for a client. It's called Dream it App (http://dreamitapp.com/). It's a platform for people to generate and discuss ideas for new web, mobile and desktop applications. If you have an idea for a cool application or if you already have built applications, add it on Dream it App and try get people talking about it.

It took about 2 and half months from start to finish, and it's a full single page application built on top of PHP, AngularJS, and Bootstrap. We learned that building large scale single page applications is much more difficult than static web applications. Static web applications refer to when the "view" (what you're seeing) is pretty much generated completely by the server side environment. Then people generally use jQuery to spice things up and make a few things dynamic. It was difficult because the javascript development practices haven't really been standardised, so we are often juggling competing best case practices. Every development company has their own opinion on javascript architecture and workflow. For example there are 5 different package managers for client side javascript: bower, ender, volo, component or jamjs. Of course there's the problem of browser incompatibility. Every browser has their own idiosyncrasies that you have to shim or monkey patch. These problems are being flattened with the introduction of Yeoman, RequireJS and Grunt, but JS development is still quite fractured. Regardless, it was great experience. One thing that tends to happen after producing something new is that you experience overruns that produce new code that can be reused in other projects. Some of these are on my gists, such as Masonry with AngularJS, Disqus with AngularJS.

Now in term 3, Polycademy has been working with Mustafa Sharara a student from Qatar on creating a new service coined SnapSearch. SnapSearch will help developers and site owners make their dynamic, interactive and HTML5 web applications indexable by search engines. Currently major search engines such as Google, Bing, Yahoo cannot index or get incorrect results from their index when they send their robot to index dynamic sites, SnapSearch will prevent that problem, and remove any economic disadvantages when using single page application frameworks. Hopefully this will usher in a world of web 3.0, where every site can dynamic and interactive. More news on SnapSearch will be posted here in the future.

In preparation for 2014, we are going to be redesigning Polycademy.com. For now, we've updated the course page with new content regarding the course content. Some of the major things include: Asynchronous Programming, Automated Development Environment Deployment using Vagrant and Chef, Build Automation using Grunt, Polyglot programming involving PHP, JS, and Ruby, Behaviour Driven Development with Codeception, Concurrrent Programming using Gearman and Redis Message Queues, and Authentication and Authorisation with PolyAuth.

But of course with so much course content, we have to be flexible depending on what kind of technology stack is suitable for the application that students want to build!

Upcoming blog posts should be more regular as I can talk about various technologies and what we're doing with them.