My language learning framework

The most important lesson that I have learned in my career of almost 15 years in software development is that programming languages are just tools to solve real life or business problems. About 5 years ago I was doing full time .NET, then I jumped to Java for about two years, then I switched to Ruby and Rails, and just recently I started using Kotlin to create two microservices (don’t ask me why, the business needed it ;)).

Because of this, there is no point whatsoever to be bound to a language/platform religiously. The languages are tools but it’s the core knowledge of computer science and software development practices that prevail.

These changes have been inspired by different rationales, sometimes influenced by the employer and sometimes by the economic conditions of the runtime environment. E.g. hosting a Java web application during 2006 was way more expensive than a PHP application, therefore, I used PHP as a primary language for my web facing pet projects. So during my career so far, I have used Java, .NET, PHP and Ruby as my core languages and also have implemented small solutions using Kotlin, Javascript, and Python.

To adapt to this changing environment of ours I have noticed I had created a framework, which I call my language learning framework. The framework itself is not about the language per se, but about the whole ecosystem around that language. It came as a result of identifying unchanging things around any language ecosystem. E.g. no matter the type of language, there is a way to deal with strings, numbers or arrays. Or the language has some sort of collections.

With almost every language that I have used, I have had the need to also learn a framework to develop web applications, a framework to do testing, and other common things around the application. So out of this experience, here I share with you my language learning framework. It lists the things I try to learn or pay attention to when I start learning a new language.

I. Language
1. Runtime & ecosystem (general knowledge)
2. Syntax
3. Data types (if there are types)
4. Main constructs
4.1. packages/modules, classes and methods/functions
4.2. loops & conditioning
4.3. arrays
5. Core libraries
5.1. String manipulations
5.2. collections (lists, arrays, maps, sets)
5.3. math (rounding, sqrt, pow, PI)
5.4. important packages/gems/etc. and any language/platform specific library
II. Frameworks
1. Application development (evaluate popular ones and pick one)
2. Testing (unit & integration)
3. Main design patterns and in that language
III. Toolset & Other
1. IDE platform specifics
2. CI/CD
3. Deployment (does the docker & Kubernetes work or is there anything platform specific)
4. Community
5. important learning resources

Some of these items on the list are dead clear so you just follow it, but some of them are hard, like choosing which framework to use. How do you choose which framework to pick. Well, in those cases, I use different inputs to make my decision but most often it is an educated guess. What I consider is, I ask my colleagues/friends if they have experience with any of the ones I am considering and get their opinions. I also evaluate possible blog posts about them and also try to measure the popularity by checking the number of contributors, how often does a major version gets released, how many job openings I find requiring that framework, and based on all these inputs I narrow my list to two options. Then I try both of them with a certain simple scenario and I try to evaluate how easily I could implement that scenario and get a feeling of both and then decide. Out of all the factors I consider, how much I like it and how many job vacancies are there requiring that framework are two criteria that weigh most with me. The pleasure to work with is very important for me, but also having an opportunity to use it (have an employer/business which actually uses the framework) is also important in my opinion.

One important thing is, when I try to jump on a new language, I try to do all these steps in a relatively short period, potentially with max 2-3 months, and then try to repeat it at least 2 times with 2-3 sample projects. In this way I make it possible for my brain to remember it, otherwise, if it takes too long to go through the list, I forget things in between and the end result is not satisfactory.

This is not a definitive list and doesn’t include everything we need to learn to be productive in one language/platform, but it certainly serves as a good starting point for me when I want to jump into a new language.

Do you have a different approach? Share with us!

ElasticSearch – Getting started

Context

A few weeks ago, our team decided to use ElasticSearch for the search functionality of the project we are implementing. I had no previous experience implementing any search engines, so I got excited to get my hands onto doing something really new. What I am going to describe you here is the process I followed do a spike on ElasticSearch and what I learned out of it.

Why should you care about it

Elasticsearch is useful on several scenarios. It is very good for searching product catalogs, creating recommendation engines, logs aggregation and searching for faster debugging, data mining and identification of patterns, and in many more scenarios. There is a chance, the product you are working on might need such a thing.

Some basic concepts first

As an entry point to this task, first I had to learn the basic concepts about ElasticSearch. In essence, ElasticSearch is a document database which uses Lucene as a search engine. It is fast and horizontally scalable, though, it is a very memory hungry system. It stores all documents inside so called Indices. An index can be imaged something like a database in the relational databases world. Inside indices, you can have types, which you might think of as database tables (not exactly like that, but a good analogy).

Data of an index is saved in one or more shards (the default is 5 but this is configurable). Every shard can have one or more replicas. A replica is just a copy of the shard and it serves for optimizing performance and failover. If you don’t have millions of data, it might be good to have fewer shards for a better performance.

Setup ElasticSearch on your computer

To start experimenting, I started with setting up an instance on my computer. Setting up ElasticSearch is pretty easy (at least for the beginning). There are numerous ways you could follow and they are very well documented in their installation page. The approach I took is to have it as a docker container on my local machine so I can start experimenting right away and for production to have a chef script to do the installation for us.

If you already have docker installed, having ElasticSearch up and running on your machine is a matter of minutes by executing pull command to get the image

docker pull docker.elastic.co/elasticsearch/elasticsearch:5.3.1

and running the container

docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.3.1

This command runs the container and exposes port ElasticSearch on port 9200. If you try in your browser executing http://localhost:9200 (username: elastic, password: changeme) you will receive a response like

{
  name: "B4VsybF",
  cluster_name: "docker-cluster",
  cluster_uuid: "EDkqSWH1Q7mjM3RSEV7kVw",
  version: {
    number: "5.3.1",
    build_hash: "5f9cf58",
    build_date: "2017-04-17T15:52:53.846Z",
    build_snapshot: false,
    lucene_version: "6.4.2"
  },
  tagline: "You Know, for Search"
}

This shows that ElasticSearch is running inside the docker container and you can start playing with it. You can find all these installation details in Elastic’s site.

Loading data to an index

Now that ElasticSearch is running, the next step is to populate some data so we can test the search functionality. At simplest, we can communicate with ElasticSearch using curl cli tool or by using a http tool like Postman, or Sense – chrome extension. Pushing data can be done manually one record at a time, or in bulk. As a first step, we could create an index by executing

PUT /documents/person
{
  "shards":2,
  "replicas":1
}

This will create index named documents with two shards and one replica per shard. It will also create a type named person.

To insert a person document, we can execute a post request with person’s data

POST /documents/person/1
{
  "name": "Arian",
  "last_name": "Celina",
  "phone": "1234567"
}

The url segments map to {index}/{type}/{id}. If we didn’t specify id, then ElasticSearch would create an id for us. The response from ElasticSearch for this request would look like

{
  "_index": "documents",
  "_type": "person",
  "_id": "1",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "created": true
}

From the response, I can read that I posted to index documents and type person and the request was successful.

For the sake of simplicity, we just posted the data to ElasticSearch without doing any data mapping. If we don’t do it, Elastic will do a dynamic mapping. If you want to enforce the type of data uploaded, you might consider to create a mapping when creating the index. Please take a look at mapping documentation for more details.

Loading more data

Posting one or two documents is easy, however, that would be of little help to really test searching functionality. To make the test more realistically, what I did was to export data from our PostgreSQL database to a json file and then posting the file to ElasticSearch using a curl command

curl -XPOST 'http://localhost:9200/documents/person' -d /path/to/file.json

If the export file is too large, you might want to split it to smaller ports (I got the right size by ‘trial and error’).

Searching

ElasticSearch offers plethora of query combinations. The query documentation is large and describes all the ways to construct complex queries with filters and aggregations. To the simplest form, for the sake of demonstration, if I want to search for the person whose phone number is ‘1234567’, the query would look like

GET /documents/person/_search?q=phone:1234567

or in a more complex form

GET /documents/person
{
  "query": {
    "match": {
      "phone": "1234567"
    }
  }
}

Either one would bring the same result. For more complex queries, please consult the documentation.

Conclusion

With such a short article, I could only touch the top of the iceberg about ElasticSearch. This article summarises the steps you can take to bring up an instance of ElasticSearch, load some data on it and start experimenting with it. Happy hacking.

My weekend with Python programming language

This weekend was a long one for me. Unfortunately, I catched cold and had to stay the whole weekend home resting and waiting to feel better. While sitting and wondering what can I do (I wasn’t in a mood to work on anything), I decided to take a look at Python programming language. It’s been a while that I have developed a kind of interest to try this language but never had a chance to take some time out of a busy schedule. Many people were telling me good words about it, how elegant it is and how easy it is to write code using it, but I needed to try it to have my own opinion. Although I could only spend 2-3 hours during the whole weekend trying it, I would like to express my first impression of this language.

What I liked

  • There are plenty of learning resources. Although I could have started with a video tutorial, I chose to start with this tutorial by google which seemed to me quite concise and straight to the point. It was showing all important points without getting too much into details. Just what I needed.
  • Coming from a programming background, I found it extremely easy to start coding immediately in python. The language syntax may seem odd at the beginning but the learning curve is smooth. It’s easy to grasp the python way of coding, although, it may take some time to forget putting the semicolon at the end of the line though 🙂
  • The language seems elegant and rich in features. It has most of the features found in other languages such as Java, C#, and PHP.
  • The fact that you can write code in the command line was interesting to me 🙂
  • Working with lists and tuple structures was extremely easy and handy.

What I did not like

  • The fact that there are no curly braces and semicolons at the end of the line might cause a little headache for those of us who come with a background from the C line of languages (C++, Java, C#, PHP, JavaScript, etc.). Omitting semicolons looked easier to me than maintaining the lining of code to keep track which code belongs to which block. After you get used to it, the whole code starts to look more elegant and easy to read.
  • You can define variable names same as built-ins overriding system variables. I think this could be a big point of confusion for newcomers.

Summary

I admit, evaluating a programming language by spending three hours learning it is extremely hard and might lead to biased judgments. However, I write this review solely because of my positive impression with this language. I like learning new programming languages as almost each of them have good parts and practices from which we all might learn and benefit. Certainly, python has its share in this and in my opinion is worthy to give a try. For me, my next step should be to find an opportunity to use it in a real life web application and see the experience 🙂

What is your experience with python? Share it with us.

 

*python logo image source: http://www.vizteams.com/wp-content/uploads/2013/08/python-logo-master.png

A beginners’ guide to web development

If you are reading this post, most probably you have some sort of interest in web development, or even you think about starting to learn about web development. In this post, I would like to show you what path you can follow to be a web developer. This is a beginners’ guide to web development from the perspective of what to learn and how to specialize. This is not a post in which you will learn coding. I just want to point out the what you need to consider before you start learning to code. So, welcome to our dynamic and ever changing world. One of those fast-pace professions with lot of challenges and excitement. So let us define some basic concepts first.

Front End vs. Back End

The initial separation you will feel here is Front End vs. Back End. Let us clarify first what is Frond End and what is Back End.

Front End

Web applications are categorized as distributed applications with a client-server architecture. So, we have a part of code which runs in the client and another part in server. The part of application which is run and rendered in client (most of the time, the client is our web browser) is called the Front End. The most usual technology combination which is used to develop for Front End is HTML+CSS+JavaScript. Front End specialists usually develop expertise in creating Front End of the web applications using these technologies. Another common skill Front End developers master is slicing Photoshop designs to HTML+CSS+JavaScript web pages.

Back End

Back End developers write code that runs on server. Usually, this part of the job entails communication with the DataBase for reading/writing data, reading/writing files, doing the business logic, etc. In some cases where the business logic resides in client side, then Back End is used to serve the data from the DataBase usually in the form of Web Services. Back End developers usually master one of web programming languages and a DataBase Management System.

 

You can master both, but from my experience, I have seen that all web developers tend to like one more than the other. Some even specialize on only one of them. Although there is a line of separation, there is no limit that which side should do what. Sometimes Front End is used only for visual representation and all the job is done in Back End. In some cases, Back End only serves the data and all the calculations and functions reside in Front End. It is a matter of design and architecture to define which side does what (although, depending on the architecture you choose, there are some guidelines about the responsibilities of each side).

Programming languages

There are a lot of available programming languages for web development. When we want to program on Front End, the defacto standard language is JavaScript. When it comes to Back End, we have plenty of choices. Let me list some of the popular choices:

  • PHP
  • JavaScript
  • Ruby on Rails (used with Ruby programming language)
  • ASP.NET (used with .net programming languages)
  • Java EE
  • Python

And this is not a definitive list, just those that came to my mind right now. So which one to choose. Well, your choice should be evaluated based on some factors like: the job market, hosting environment of the web application, available learning resources, available time to learn, the development community around you.

If you want to work as a web developer, in my opinion the most important factor is the job market. You should analyze the job market you are in (or you want to be in) and chose that language that has most job openings. Another important factor is the hosting environment. For example, PHP hosting is quite cheap compared to Java hosting. If you are going to develop an intranet application which is going to be hosted internally in an organization, perhaps Java EE could be a very good choice, but if you want to host your application online, Java EE could be rather expensive compared to other languages.

With the popularity of Node.js, JavaScript has started to become a popular choice of Back End programmers, however, this is still quite a new and immature technology compared to others, and I would not recommend it as a choice of beginner Web Developer.

In my opinion, PHP has the easiest learning curve, cheap hosting environment, plenty of learning resources and relatively easy development environment, so I would recommend to any beginner web developer start with PHP. ASP.NET is also a good choice. Microsoft offers a lot of learning resources, free development tools and a pretty rich environment. If you like the Microsoft ecosystem, ASP.NET is a very good choice.

Frameworks

If you are a beginner, give yourself some time before you start learning a framework. Frameworks are code libraries which make the life of a web developer easier. Frameworks give a structure to a web application, help web developer do some tasks a lot easier and faster then coding everything yourself. If you want to be a professional Web Developer, then it is a must you learn at least one framework, which boosts your speed of development.

You have a plenty of frameworks which try to be general solutions or specialist solutions. You must evaluate your needs. If you have chosen PHP, I would recommend Laravel as a framework of choice. It is a sound MVC framework which is quite trendy these days. If your choice is with ASP.NET, I would definitely recommend you learn ASP.NET MVC and EntityFramework at least.

Web development can be huge and you may want to focus on one type of applications, let’s say development of web sites with Content Management Systems (CMS). Again if you have chosen PHP, I would recommend you continue with WordPress. WordPress allows you to create web sites, blogs, but also it can be extended with ready plugins or custom themes and plugins to quite complex business applications.

You will find plenty of choices for frameworks for any language you choose, so based on your language of choice, you will have to work with different frameworks.

What next

As a first advice, even if you choose to specialize for Front End or Back End (I would strongly recommend you do), you should have a grasp of the other side, and if you do, your team’s performance will be better. If you have learned a programming language and mastered a framework, what I would recommend is you start with another one. Programming languages have their own philosophies and paradigms, and sometimes some differ quite a lot. Knowing two or more programming languages will allow you have a better picture and understanding how programming problems are tackled and will make you a more fluent developer. As I said earlier, you have to consider many factors when you choose your languages. My choices until today were: JavaScript, PHP, ASP.NET, and Java EE. I’m still looking forward to extend my list 🙂

Things every programmer should know

Learning process of developers never ends. It is in our DNA to continuously feel the need to learn and self develop. As much as we learn, we have even more to learn. The technology world is moving so fast that to keep the pace and be up to date with cutting edge knowledge, we need to embrace learning as part of our daily activities. Whenever we say we learned to use a framework, we see that 7 other frameworks have become famous and we need to take a look on them. One of the most frequent questions of my students is the question “what should I learn to be a good programmer?”.

Luckily, nowadays, learning resources are so vast, we have difficulties to chose which one to pay attention first. We have plenty of books available (free and paid), free magazines, blogs, video tutorials, MOOC courses, etc. However, cutting the line what one should learn to be a good developer is not an easy task. As doctors have chosen to specialize only on one part of the body, developers too need to decide for a specialization, be it system developer, web developer, business application developer, etc. The specialization is something which comes to question when you reach a steady state of development knowledge and experience, but what do you need to learn to get there. What could be a good developer knowledge path?

Over the time, I have come to conclusion that developers need to know the whole stack of code execution in order to succeed.

What do I mean with this? The computer literacy starts with using the computer and perhaps some basic configuration and application usage knowledge. If you advance in configuration skills, you learn perhaps also about servers, monitoring, user management, etc., you start learning system administration. Then you need to network the computers and servers to work together, and start learning networking, which builds on top of administration skills. Then you feel the need of automation, and that’s where you start with learning programming.

Why do I think this model works? In early times of software development, we used to have only desktop applications which performed certain functions. Today the whole view is different. We rarely develop single computer applications. We either create a web application or have a network aware app. Those applications often may face problems of different nature, especially if they are running inside a corporate infrastructure where proxies, firewalls, and other infrastructure components interfere with the traffic and environment. Most of the time, problems caused by those components will not demonstrate themselves very clearly and often it will look like the application is failing. Finding the root cause of the problem requires good troubleshooting skills and that requires understanding of system administration and networking which are the layers above which your code runs. If you fail to have the understanding of underlying platform, you will be left alone to find what is the problem with your code, as most of the time, system administrators/network administrators will not be able to identify why your application is failing (and perhaps it is not their fault as they do not know the ins and outs of your application), therefore the judgement will be that the code is faulty 🙂

What should you focus to learn? Do you need to be a system administrator and network administrator before you start to learn programming? No, I am not saying that. My suggestion is you get at least the basic understanding of operating systems (perhaps not all of them but those which are in your environment), learn how user management is working, the privileges, and the connection to central user management like Active Directory. Learn how to read system logs, that is the place you will most often find valuable information about what is happening with your code and find the clues about what is going wrong, how to list running processes and see who is consuming the memory and who is consuming the processor. Learn a little bit also about system security,  to setup and configure your production environment and its subtleties and the basics of how the computer networks operate. In the long run, you shall benefit from this knowledge for sure.

I do believe that a “complete developer” is not the one who only knows to write code and nothing else. You should know the whole picture of the environment you develop and the environment your code runs, and this requires you to get a little bit out of your zone and see what you have around. I would like to explicitly clarify one point here. I do not think that developers are above or more valuable than System Administrators or Network Administrators. Everybody in this ecosystem has his/her importance and value. I just want to point out that if you want to know the full stack of execution of your code, you have to have a clue what happens beneath and these are the things every programmer should know. Troubleshooting problems of your applications will most likely come to you as the last line of support, and finding faults in system or network requires basic understanding how those function.

What do you do when your code has cancer?

One truth about business software applications is that they evolve together with business requirements. From different reasons, sometimes from lack of experience of developers and sometimes from not having enough time to devise good solutions, the code often tends to create some sort of cancer inside, which progressively grows to unmanageable pieces of software until it breaks the whole system.

What do we do if we find such phenomenon in your code?

In my experience I have seen that dealing with such code is almost inevitable in brown field projects. Ignoring them makes things only worse. You have to face it and treat it as it is a disease which needs to be cured. Whenever I face such a situation, I follow these steps:

1. Isolate

Changing the code without breaking the whole system, or refactoring as is called in software development terminology, is not an easy task. Every change must be validated and tested well to ensure that changes do not break or alter the current functionality. In order to stop the bad code growing, we need to isolate it first. Isolation could be done by aiming abstraction through loose coupling the part which needs to be isolated. The abstraction is done by creating interfaces for every class that needs to be isolated and make dependent classes call to interfaces instead of real classes. This may require introduction of inversion of control (IoC) / Dependency Injection (DI) container. Each isolated part must be supported by quality unit tests which ensure that the abstraction maintains the code logic which was in place. Unit tests will also help us in the second phase.

2. Repair

This phase is quite simple. When you arrive here you have isolated the bad code and protect the functionality with unit tests. Now you can safely start to refactor and change the bad code. As you change, you should continuously run the unit tests to check if changes have any unexpected effects. If the unit test coverage is OK and they are written well, then you can safely proceed as unit tests will spot if you have broken something unintentionally. I would highly recommend Test Driven Development in this phase.

3. Integrate

Integrating the isolated code is the last phase, and most probably the easiest. Now the bad code should be gone and the functionality is unchanged. Often you shall find this phase unnecessary as the integration has been done during the phase 2. However, sometimes you will find some kind of double abstractions resulting from a lot of refactorings and you will see the opportunity of cutting these unnecessary code.

Conclusions

Perhaps this strategy looks very simple, but in my experience, it has been a very effective one. We are not always lucky to land in green field projects and define good code from the beginning. Besides, it is not always easy to maintain the code in good shape when the team has many developers and there is not a good development process in place.  Sometimes you deliberately allow the code to go in bad direction as a compromise to speed or resources.

Web services vs. SOA and pretty URL vs. REST

It has been quite a time since Service Oriented Architecture (SOA) and Representational State Transfer (REST) architectures are around, yet there are some misconceptions about them I hear very often, which I would like to discuss here.

1. Having web services does not mean you have an SOA architecture

This is perhaps one of the biggest misconceptions about SOA architecture I hear very very often. I see many developers thinking that if they have a web service or two in their architecture, they say their architecture is an SOA architecture. I think this comes because of two reasons: 1) “Web service” and “service oriented” resemblance in naming makes people think they are the same thing; 2) As web services are the most common way of implementing an SOA architecture, this pushes people think that when they have created one web service, their architecture is an SOA architecture.

An SOA architecture is characterized of composition of independent services which encapsulate business functionality and expose it as a service, which can be  a web service, a windows service,  or any other form of exposure. Ubiquity of web and advancement of web development technologies which made the creation of web services easier have put web services as a mean of choice for implementation of an SOA architecture, however, the definition of a service within an SOA architecture does not put web services in any special position regarding implementation of SOA architectures. Here is the definition of a service from Open Group:

A service:

Is a logical representation of a repeatable business activity that has a specified outcome (e.g., check customer credit, provide weather data, consolidate drilling reports)
Is self-contained
May be composed of other services
Is a “black box” to consumers of the service

 Web services and SOA make a great pair which power a lot of important and popular service out there, but it should be crystal clear to Software Architects, Developers, and anyone contributing to software developments that having web services does not mean you have an SOA architecture.

2. Having pretty URLs does not mean you have a REST architectural style

REST architectural style is another popular topic lately, and as such, is subject to a lot of misconceptions as well. REST has brought simplicity to implementation of web services and is embraced very popularly from the web development community. It plays well with the HTTP protocol, which we are familiar with ever since the beginning of www era.

One characteristic of REST architectural style is that resources are at the center of the architecture, and they are beautifully represented in URLs. REST has brought us pretty URLs, and therefore people have created a connection between the URLs and REST architecture. Leonard Richardson has developed a maturity model which tells the level of your API or RESTful services to what degree are RESTful. Martin Fowler has a great post about richardson maturity model. The levels are described as in this picture:

richardson's rest levels of maturity for REST

The very first level of the pyramid, level 1 is the implementation of URI or introduction of resources and at this level of implementation resides using the “pretty” URLs to connect URIs to resources. The level 2 refers to using HTTP verbs and the most advanced level, level 3 refers to using hypermedia or following HATEOAS.

In Richardson’s model is clearly seen that pretty URLs are the very basic level of implementing RESTful web services towards a REST architecture and clearly explain that having pretty URLs to access your services does not mean you have a REST architectural style in place.

In my opinion, implementing an SOA architecture and RESTful web services requires to have a clear understanding of these concepts, the constraints which come with them, and what is required to have such an architecture in place. Clear understanding will lead to effective architectures which allow us to reach the promises and benefits of these architectural styles.

Architecture of web applications

I consider software development more art than exact science, and as such, in software development almost always there is not a single way of solving a problem. Although there are defined best practices, it is a matter of problem being solved and the knowledge of the team that influences most the definition of the architecture of web applications and software applications in general.

Recent years has brought to popularity using REST in the architecture of web application solutions. I am a huge fan of REST, I like it a lot mostly because of its consistent way of expressing CRUD operations and brings simplicity to the API implementation. The use of REST has been pushed further with advancement of MV* JavaScript frameworks as they tend to have a natural way of consuming resources from REST APIs.

Lately, I am seeing that most newly built web applications tend to use REST in some way, if not exposing APIs, they do consume one or more of them. In my opinion, REST tends to create a viral effect on developers, as much as you use it, you want more of it. Now after we experience REST, I think there is a question which pops up:

Do we have to expose everything as REST API?

The universal answer “it depends” applies here very well! If you want to use a client side framework such as AngularJS that does not need server-side code and is well suitable to consume REST services, it might be a very good idea to expose the whole business logic as REST API and consume it through AngularJS services. This comes with additional benefit that if you want to have a mobile app of your web application, you do not have to write any additional code on server-side to support your mobile app, just consume the very same REST services and you are good to go. A diagram representation of such layered architecture of web applications could look like this:

Web application architecture with REST API and client side MV*

This is one of the most often used architecture styles I am seeing these days in web applications. However, what if you are not keen to using MV* JavaScript frameworks and want to use a server-side backend and generate your HTML representation using let’s say ASP.NET MVC. Do you still need to expose business logic as REST API?

Exposing business logic as REST API or any other form of service layer, if you do not have multiple types of consumers (web, devices, etc.), in my opinion is waste of resources (time and effort). Exposing and consuming services do introduce a level of complexity (you need to put extra effort on error handling, security, versioning, asynchronous access, etc.)  to the architecture and application code. This complexity is non considerable if you have multiple consumers of your application as it avoids writing multiple times the same functionality, however, if there is only one consumer and it is the web GUI, then in my experience I have seen that it only makes things worse. In such a situation, going with an old style server-side backend is a lot more easier. If you say I have a mobile client as well which consumes part of the functionality, in that case I think it is better to create a small REST API service group exposing only that part of the functionality, is a better choice. This will also allow you to develop your web app and mobile app in different paces (if you have lack of developer resources, you shall come to this requirement). The modified version of our new architecture will look like this:

Web application architecture with server-side backend and rest api

Sometimes we are keen to jump to new technologies and architecture styles just because its presentation looks attractive and we like to get our hands in it and give it a try. Often this pushes us to situations which makes maintenance of our code base difficult in later stages of our application lifespan. It is very important that evaluation of the application architecture to be done as early as possible and be judged from the simplicity and maintainability perspective.

 

Offline availability with AppCache

Offline availability of your web application will give a little extra comfort to your users. If using Local storage an appropriate choice to enable web applications to work offline with dynamic data (see my post on Local storage), Application cache (hereafter AppCache) is a good choice for making the static content of the web application available offline.

About AppCache

AppCache enables developers to specify a set of files which need to be cached by the browser. While cached, the requested resources will be served from the disk and this will make your web application available even when users are offline.

Caching static content has several benefits which you may like to leverage, including:

  • Better site performance, as static content will be served from the disk and not requested from the server
  • Better server performance, as less resources will be served and bandwidth will be preserved
  • Users will have the possibility to access your contents while offline, leading to better user experience and satisfaction
  • Mobile users can use the web app even when they are out of internet coverage
  • Mobile user can bookmark/pin the web application to their phone and use it later as a mobile app (offline)

The cache size limit may vary between browsers, but a cache size of 5MB can be expected from modern browsers, and with users permission, the size can be increased if needed.

How do we tell the browser to cache our site, and most importantly, which files to cache. We can do this using a special file called Manifest file. We tie manifest file to the html root element using manifest attribute. e.g.

<html manifest="webapp.appcache">

It is recommended that the extension of the file is .appcache.

Manifest file structure

Manifest file is a simple text file which should be served with “text/cache-manifest” content type. The very first line of the file should be:

CACHE-MANIFEST

and after that files to be cached are listed one per line.

index.html
css/main.css
css/theme.css
js/lib.js

Folders and wildcard specifiers are also permitted.

The common practice is to follow this structure:

CACHE MANIFEST
#23.10.2014 v1.0.10
/file1.html
/file2.jpg

NETWORK:
posts.php

FALLBACK:
offline.html

On the first line we have put the mandatory declaration of “CACHE MANIFEST”. Then on the second line we put date and version as comment. After version, we start declaring files/folders which are to be cached. The “NETWORK” section shows which files should not be cached as they change frequently, and last, with “FALLBACK” we specify what should be used when a resource is unreachable.

The comment part with version number has a significant importance here. The browser will not ask for the cached files unless one of these two is true:

  • The cache is deleted
  • The manifest file has changed

We best note the change of the manifest file by putting a version number in it. Whenever we do a change in any of the cached files, we simply increment the version number to advise the browsers to re-cache the files.

Specifying the manifest file will make the browsers cache the content specified and make the static content of your web app available offline.

 

HTML5 Local Storage API

It has started to become a common requirement for Web applications to have some support for offline usage. HTML 5 local storage (hereafter local storage) is one of the options to consider when you need to support offline use of your web application. But what is local storage, and when should you consider it. Let’s go through what you need to know before you start using local storage.

What is HTML5 Local storage API?

Local storage is a storage of key-value pairs (KVP) which you can store and access using a JavaScript API. It is supported by most of major browsers (even IE8 does support it :)). The KVPs are string values of information used to store in the storage of the browser

e.g.  “CMS” => “WordPress”

Local storage has several advantages over Cookies and can be a good substitute for them. Cookies have the size limitation of 4KB, on the other side, the usual supported size for Local storage is 5MB. Also, cookies are sent to the server with every request, overloading the bandwidth, whilst, Local storage is not sent to the server any request, instead, only the required value is send.

Local storage is a persisted storage and have no expiration time. The data persisted will stay until cleared. There is another variant of browser storage which is temporary, and that is Session storage. Session storage is used only for the session, and when session expires or is destroyed, so is the session storage.

Storing and reading from Local storage

Before starting to use the Local storage, it is advisable you always test first if it is supported by the browser, and you do it with a simple check like this:

if (window['localStorage'] !== null) {
//your code here
}

Storing values to Local storage is done using setItem() function. This method accepts two parameters, the first one is the key, and second parameter is the value. e.g.

localStorage.setItem("cms", "WordPress");

If we open chrome developers tool in resources tab, we can see the values in local storage, and there is our “cms” key with value set to “WordPress”

Local storage in chrome developer tools

 

Reading the stored value is done using getItem() function and supplying the key as a parameter. e.g.

var cmsValue = localStorage.getItem("cms");

Other functions

There are several other useful functions which you can use with Local storage. If you want to get the total number of keys stored you can use localStorage.length property, or if you want to remove one item from the storage you can do this by calling removeItem function. e.g.

var numberOfItems = localStorage.length;

localStorage.removeItem("cms");

It is also possible that you clear the whole list of items stored by calling function clear e.g.

localStorage.clear();

Events

In some more advanced scenarios, you may want to track changes in local storage, that is where events come in. Events are out of scope of this post, however, it is good to remember that they are there should you have such a requirement.

References

If you want to get in details, the specification of the local storage can be found in this link: Web Storage specification