Dear reader, I’m writing the article to present my first ever web application to go live; «companywatchlist.com». I will both tell my story learning about web development and also about the technologies and inner workings of the site. I would like to give insight into what it takes to get into web development and create an application using modern technologies and techniques.
The last year I have spent hundreds of hours (you may even run out of fingers on your second hand) trying to learn web development. It has been one of the most challenging intellectual pursuits I have ever undertaken. At times, I doubted the value of my struggles, but now looking there is no doubt in my mind; I would do it all over again.
Before I continue I would like to quickly introduce myself. My name is Didrik Fleischer, I am at the time of writing 25 years old. I’m from Oslo, but living in Sandnes, Norway. I’m a second lieutenant in the Norwegian army, and I’m working as a LAN technician in NATO Communication and Information Agency.
I have always found computer programming fascinating; it seemed so mysterious how programmers managed to create everything from massive virtual worlds in computer games, to worldwide communication platforms and online banking systems. Being an outsider to the world of programming, a lot of the mystery I felt was probably a result of the mind-boggling complexity I prescribed to the field after looking at my first pages of code. Through the years doing my bachelor in Telematics I had been exposed to many other parts to the puzzle of how input gets transformed into output on a computer. Physics, electronics, telematics, computer networking, operating systems and much more, which both resolved and added to the mystery of how programmers were able to create the vast ecosystem of applications we use every day. To be honest, going through school and the first few years working, I felt that programming was beyond me. “My time is better spent doing something a bit simpler”, I thought to myself. I don’t think I’m alone in this line of reasoning, maybe this article can push you just a little bit further towards staring to learn programming, if that is something you are interested in doing.
Working as a LAN technician I got heavily exposed to Cisco and networking technologies. I felt that in order to be a valuable employee, I should go through some of the Cisco certification tracks. Fresh out of school I hungered to learn some real job relevant skills to kick off my career. So I completed the CCNA and CCNP Routing and Switching certification tracks. Through work I also got exposed to the compute, storage and virtualization domains as well as Microsoft enterprise technologies. I grew more confident each day in my ability to understand information technology. It really was this confidence that enabled me to make the decision and commit to learning more about programming.
Another force pushing me towards programming was the feeling that without programming knowledge I would forever be doomed to subpar and unfulfilling understanding of the very field I am building my career around (IT). This was simply not an option for me.
So why did I choose web development, why not mobile apps, desktop applications or micro controllers? I think primarily what got me interested was:
- The massive reach I could have through the internet and the ease of use associated with the web browser. An application could be deployed, and immediately be available to everyone throughout the word (given they got an internet connection). No installation would be required by the users, it would be as simple as navigating to a URL.
- A product could be launched and be able to deliver service across all platforms (phones, tablets, Mac’s, PC’s), this would not be the case if decided to go for C++ or swift.
- I also knew that there were libraries that allowed you to build cross-platform desktop applications using nothing but web technologies, which also intrigued me.
- The web browser as a platform also seemed sufficiently powerful, despite the additional layer between code and hardware as well as JavaScript being a scripting language.
If you are aiming for a career as a programmer, or intent to spend a lot of time on it, what languages you start with does not really matter. All programming languages are very similar. Most, if not all support functions, conditional statements, loops, object oriented coding principles, some way of dealing with exceptions, and common datatypes like int, float, string, arrays etc. My point is; if you are a proficient in one language you will quickly pick up another.
Now as you know a bit about how I got into programming, and web development more specifically, let me continue by explaining how I went about learning all this stuff. Where does one begin and what resources are available? After having gone through the Cisco certification tracks I had learned to appreciate good learning resources. When you are going to put the time and effort into learning something, you want your learning process to be as painless and effective as possible. If you get access to better learning material if you pay a little bit for it, you should do it. Your education and career are worth investing in, period.
When it comes to learning programming, I would start with a book or video course on the basics of the respective language you have chosen. Knowing the basics well is very important, and will make it a lot easier to grasp the more difficult concepts later. For me, much of the path formed as I was learning. When I discovered new holes in my knowledge, I would try to fill it or take a note of it, so that I could return to the topic at a later time.
Do remember to actively practice by solving coding problems on your own, just reading or watching videos is not enough. A fantastic place to practice solving coding problems is “codewars.com”, sign up and start coding. Although I have not used these, Learningtree and others alike provide great introduction to coding with an interactive learning platform.
When you have some legs to stand on, I recommend coming up with some projects of your own and trying to build them from start to finish. If you have a hard time coming up with something, or you would like to have some guidance along the way, there are courses available that will hold your hand as you build an application form scratch. This is great to do in the beginning, when you don’t really know how to put it all together yet. If you want to learn PHP, I found “codecourse.com” to provide exactly this.
I do not have a comprehensive map of all resources available to you when learning about web development, but I can list the main sites I used:
- Safaribooksonline.com
- Lynda.com
- Codecourse.com
- Laracasts.com
- PHP.net
- Developer.Mozilla.org
- W3schools.com
- Laravel.com
- Vuejs.org
- Codewars.com
- Stackoverflow.com
In addition to these sites I have read hundreds of articles on various topics around the web, these are hard to document with references. But keep in mind that for pretty much every problem you can articulate, google will provide links to an article or forum thread about exactly that.
I cannot leave this section without extending my gratitude for the massive help of the stackoverflow community. If you have a problem, someone else has had it before you. And most likely you will not only find help, but also code examples on stackoverflow . It’s a priceless resource for all developers.
Before diving into the technologies involved in building a site like Companywatchlist.com, I would like to talk about the different features of the application. There are four main features the site is offering.
- The first is the ability to search for a company by name or ticker and be able to access a web page showing some financial data on the company.
- The second is the ability to capture your thoughts on a company by filling out a simple analysis form. This analysis form is not representative of how one perform financial analysis or valuation of a company. It is just ment as a "proof of concept".
- The third feature, and also the most central feature to the site, is the ability to organize the companies your are following into watch-lists. You can create as many watch-lists you want, and add as many companies you want to each watch-list.
- The forth feature, and the most challenging to write, is the ability to create custom notifications. A notification consists of one or more conditions you combine together, and when all the conditions are met you are made aware through email and the web GUI. An example of a condition could be; the stock price is below $85, or the P/E ratio is below 12.
You can read more about how companywatchlist.com works and its faetures at companywatchlist.com/about.
The following section will be on the different technologies involved in “companywatchlist.com”. This is no comprehensive introduction to any of them, but more of an overview on what goes into a simple web application these days. I think I will tackle this by starting on the server side, moving from hardware and working my way through the operating system and webserver software. After that; the server side code and supporting services. Then moving to the client side; talking about the technologies in the browser. And finally the services and applications not directly involved in creating the functionality of the web site, but rather supporting the development process. You may use the links provided throughout the section if you want to read up more on what I am merely mentioning here.
Hosing: Amazon Web Services
To host the project I’m using Amazon Web Services, using a t2.micro (Variable ECUs, 1 vCPUs, 2.5 GHz, Intel Xeon Family, 1 GiB memory, EBS only). Which should be capable of thousands of daily visitors; complete overkill for my usage.
Server OS: Linux
The server is running Linux (4.4.0-21-generic GNU/Linux). Since the project is using the Laravel PHP framework, I found it very helpful to utilize Laravel Homestead during development. Homestead is a an official, pre-packaged Vagrant box, providing an environment for PHP development without worrying about installing php, webserver software, database and all the other things needed just to get started with coding. Using a VM like this also keeps you from messing up your operating system with all sorts of software.
HTTP server: NginX
I’m using NginX as HTTP server. The HTTP server is responsible for accepting and responding to HTTP requests as well as maintaining the HTTP session.
The server side code is written in PHP using the Laravel framework. Laravel is today one of the most popular frameworks in the PHP world and are getting more and more traction. The framework is mature and feature rich. It solves pretty much all common PHP development problems, and does so while still being flexible enough for you not to feel too restricted in terms of what technologies you want to use. Due to its popularity many services provide easily installable drivers and plugins, allowing you as a developer to quickly integrate them.
Going off on a tangent; when feeling fairly confident with PHP I would recommend learning a framework to take your coding skills to the next level. Let’s be honest; coding at a large scale is diffi-cult, and taking advantage of community projects just makes sense. It allows you to do more, better, in a shorter amount of time. Reading other people’s code is also a huge source of learning. When it comes to frameworks they can teach you a bunch about development in you respective language. They provide good solutions to common problems, shows you ways in which you can structure your codebase and help you make your code scalable, maintainable and easier to collaborate around (given you all know the framework). Don’t get me wrong, frameworks are not suitable in every situation. If you can handle the job with some simple static pages or dish out the wanted functionality quickly with plain PHP you should do so. It is when the project becomes larger that a framework comes in handy.
Database: MySQL
I’m using MySQL; the hugely popular open source relational database server. Working with MySQL requires you to learn SQL (Structured Query Language), relational database fundamentals (design and normalization), how to interact with the database server through PHP and much more. It’s a whole area of expertise, like most other things in IT.
More about Laravel
A lot if the time spent developing “companywatchlist.com” was spent in the Laravel documentation. Although I’m no authority on Laravel and I still have huge amounts to learn, I’d like to attempt to write a few paragraphs on the core components of the framework.
Laravel is MVC framework. MVC stands for “Model View Controller”, which is a software design pattern, providing a solution on how to structure you application, its logic and components. It has three main parts; the model, the view and the controller. The model manages the data, logic and rules of the application. It is responsible for retrieving and persisting data according to commands from the controllers. The retrieved data is passed to the views which in turn render the output to the user. The view is the generated output of you application. The views are invoked by the controller that is handling the current request. Data from models can dynamically adapt the view based on the request. You define a single view which then dynamically displays content and personalize the experience for each user. The controller handles the requests by accepts inputs and converts it into commands for the models and views. Examples of such commands can be telling a model to update some data and have the view render that updated data accordingly.
Read more here:
https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller
http://laravelbook.com/laravel-architecture/
The service container is a core component to almost all larger object oriented codebases these days (not sure how correct I am in this statement, but this is my impression). It took me quite some time to get my head around it, and some parts of it are still fuzzy. Service containers are one of those things I just could not understand before I tried to create one of my own. Sometimes it is like that; you simply have to do it yourself. Let me try to explain.
Problems arise when building large projects that are going to stick around for a while. The problems I am referring to are testability and flexibility. You want to be able to test the components of you application separately to verify their doing their job correctly. You also don’t want to lock your objects to specific implementations of the services they are depending on (the design principle stating you should code towards interfaces rather than implementations; look it up).
To keep it dead simple the service container allows you to register objects into a container (object, array or other data structure able to hold key-value pairs) and resolve objects out of the container. By building your objects in this way you can intercept the process of how objects in your application are built and resolved. You can simply swap out the implementation of a service (object), and have that change reflected throughout all other objects depending on it. Service containers give power to control your dependencies and the dependency injection process. This makes testing of your code a lot simples as you are able to mock dummy objects mimicking the behavior of the real thing, allowing you to test individual objects and functions without relying on your dependencies also to do their job correctly. The testing of one object does not depend on the functioning of other objects, obviously useful in many cases. A turning point for me, trying to understand dependency injection was to learn about the reflection API in PHP. The reflection API adds to the ability to reverse-engineer classes, interfaces, functions and more. This information can then inform the process of how objects are created.
I’m going to stop here but feel free to keep reading; for instance at php-di.org.
Service providers confused me for a long time, and still do, and like with many other things in programming I will stay confused until I try to create the functionality on my own. Service providers come into the picture when talking about application bootstrapping. Bootstrapping is in general the process of registering things in your application and making them ready to handle the incoming request. These things could be service container bindings, event listeners, middleware and also routes.
Service providers are the place to do all of you application bootstrapping. Even much of the Laravel core is bootstrapped using service providers. An example of this is the router:
In the end, the router is an object; an object build from a class, which might extend another class, using any number of traits and all of these probably have a number of dependencies. Before the router is ready to be used by the application to handle an incoming request, the router object must instantiated and registered with the service container. This is the kind of thing that happens during the bootstrapping process, and this is the kind of thing we do in service providers.
Read about Laravel service providers.
I’m going to be fancy here: “Facades are static proxies to underlying classis in the IoC container”. These are not my words, but they do boil it down very well. This statement I would presume is more confusing than clarifying to someone not intimately familiar with the terminology used. I’ll try to explain:
A facade is a class that provides access to an object in the service container. When calling a method on a façade, behind the scenes that object is resolved out of the service container and the called method is executed. The façade uses the PHP magic method “__callStatic() to defer calls from the façade to an object resolved from the container. The façade pattern can also be taken even further by creating a very simple interface towards poorly designed or complex APIs.
Facades just make things easier, nicer and prettier to work with. Superficial you might say, but developer happiness is important. Developer happiness is reason enough to put the effort in and provide facades to your services.
Continue reading at the Laravel documentation, or an article on sitepoint.
A contract in Laravel refers to, in PHP terminology, an interface. Laravel uses contracts to define the core service providers of the framework. If you adhere to a contract you should be able to create an implementation that could replace the existing one. Discussing contract/interfaces leads us into the design consideration of writing loosely coupled code. Loosely coupled code means that the classes we create do not depend on any particular objects. We instead write our classes to have dependencies expressed through interfaces. Any object that adheres to an interface is a valid implementation of that interface and can be trusted to perform the work promised by the interface.
Let’s say during development of an application the need to log error messages arises. The developer sees no reason why this should be more advanced than dumping the error messages to a text file, but she does realize that this my change in the future. The developer therefor first create an interface describing what an error logger should be able to do, and then creates an implementation of it that logs error messages to a text file.
Later down the line, our developer sees that the application need to log error messages to a database instead, in order to be able to analyze the error logs more easily and extensively. In a brief moment of despair our developer realizes that a bunch of classes are relying on the ability to log errors and it would be cumbersome to track them all down to update them one by one. But then she remembers that she created an interface to represent the error logger and that this interface is what the classes in her application are depending on. She should therefore be able to make a new implementation of the error logger interface and then update the binding in the IoC container accordingly to solve the problem. There is no need to update the dependencies of her classes. The power of the IoC container becomes apparent here. It is not the class’s responsibility to know how to get ahold of its dependencies. The IoC container is the one building new objects in the application.
Read more about Laravel Contracts.
The router in a web application gives you the power to fully customize the interface towards your application and ability to control what files, classes and methods they point to. This is the sort of thing that would be overkill when only having a few static pages, but with a larger project that will evolve over time it is extremely useful to restructure your URLs or have them point other places. Just imagine you are updating a feature of you app, but you are not quite ready to redirect all traffic towards this new and largely untested implementation. With the power of a router you may have your route point to the old implementation until you are ready to swap out the old with the new, and this could be accomplished simply by updating the destination of the existing route to the new and improved solution. There are many other use cases; this was just to give an example.
The Laravel router also enables you to discriminate between routes based on HTTP method and route parameter (which can be defined as optional or required). Routes can also be renamed, to make it easier to refer to them throughout your application. Some of the more advanced features of the Laravel router are the ability to send requests and responses through “middleware”, before and after a request has been handles by the application (see below). There is also Route Model Binding, meaning that Laravel are able to retrieve and inject models that correspond with user provided route parameters. This resolution logic you can largely leave untouched, as Laravel will resolve the corresponding model when your parameter name is equal to the model name you are type hinting in your class constructor or method.
One of the things that demystified the concept of a router for me was that at its core it really is just matching the incoming URI with a class/controller and method. This class is resolved through the Service Container, which after all the crazy reflection and object building logic uses require/include to pull in files (well, not exactly, it uses the composer autoloader, but still; it’s pulling in files and composing their contents together).
There are more features I have not mentioned, but it will do for this article. Read more about routing in Laravel.
Middleware provides a convenient mechanism for filtering requests before entering your application. If any middleware are defined, ether for all requests entering the application or the specific route being hit, the request must go through it before being allowed to proceed further into the application. This is for instance how checking if a user is authenticated is handled in Laravel, it’s done in middleware. This allows for logic that redirects unauthenticated users to the login screen while others are allowed to proceed to the intended controller.
Middleware could be chained together to sift out only the requests “worthy” of being handled by your controllers. Read more about middleware in the Laravel documentation.
Controllers gather related request handling logic into a single class. You could define all the logic in closures in your routes-file (the file where you define your applications web routes), but this would quickly become very messy.
I feel like it’s not that much to say about controllers (although it probably is). What each controller does is very dependent on your application. If I were to point out some common characteristics; they usually take the data passed from the client, do something with that data, like persisting it to a database (room for all sorts of logic here of course) and create the appropriate response, whether that means to return a full blown view (HTML, CSS and JS), or just a simple JSON response.
Laravel request objects makes it easier to work with the incomming request data and headers. Instead of having to work directly with the $_GET, $_POST and other super globals PHP stores HTTP request information in, Laravel elegantly wraps this up and makes it easily accassable through the Illuminate\Http\Request class.
As requests come in to our application there are many concerns to consider: security, data integrity, data validity, authentication and authorization and much more. Some of which is discussed below.
Validation can be easy to overlook, why can’t we just assume the users will give me the data that I ask for and nothing else? It’s also kind of boring, uninteresting and cumbersome. It does not add functionality, yet it is hugely important. Users will not always give you what you expect, and some people even have malicious intents and will actively try to break your site. We need to implement means of protection as well as making it as easy as we can for the user to understand what went wrong with her request. Laravel provides an easy to use method for validation, which can also be extended to add you own custom validation logic. Say hello to conditional statements and comparison operators, they are your friend when it comes to validation. But do bear in mind, if you are working in PHP, which is a loosely typed language, behind the scenes variables can change types and comparison operators might not work as you expected. One tip I can give is to always use “===” to make sure data you are comparing also are of the same type.
Laravel provides several different ways to validate incoming data, read more about them here.
To make sure you are not allowing users to execute code on your server, or perform Cross Site Scripting (XSS) attacks; input processed by your server should be sanitized. There is a number of ways user input might end up being interpreted as legitimate PHP code, at which point “you are screwed” and it only needs to happen once throughout your entire application. I am no software security expert, but all web applications should implement some level of input sanitation, and to make it easy to not forget, it should not be implemented in a controller, but rather as early as possible in you code, like in middleware would be a much better place to put it. But let’s be real, even though you have done everything in you power to make sure no unwanted code has been executed in your app or persisted to your database, it is still possible that you have missed something. You should therefor also always escape all output in your views. This means, when working with HTML, to remove all html tags. By doing this you make it impossible for an attacker to output "<script></script>" tags into other people’s browsers.
There are loads more to this topic, but I’ll leave it here. You can read more about PHP security at OWASP.
The response from a web application can be many things, but in 99% of the cases we are talking HTML, CSS and JavaScript or an XML or JSON object. Laravel's response objects allows you to easily manipulate the HTTP responses from your application. You can make it as simple as returning a string or array in your web/routes file and have Laravel automatically generate the HTTP response. Or you can take control yourself and easily add headers and cookies, and return blade template views, JSON objects or files.
In Laravel the Blade templating engine comes with the framework out of the box. This does all sorts of useful things, like allowing you to extend other views which makes it possible to make one base view containing the head, navigation and footer, elements that are going to be on every page anyway. This makes it so much simpler to make changes to these reoccurring elements. Blade also does output escaping automatically for you. There are tons of awesome features to the Blade templating engine, I’ll leave this section like I do the rest; with a link (Laravel docs on Blade).
Laravel makes integrating a login and registration system in your application a breeze. It basically auto generates all the database migrations and markup needed to have a functioning login and registration system. I have added my own twist to these (introducing the concept of administrators and users needing to be accepted before being able to log in), but the auto generated code is largely untouched. Creating a login and registration system is something that darn near every web application needs, and having to create it yourself is a bit of a pain. But Laravel to the rescue; making it so simple.
When cooperating with other developers or use multiple platforms during the development process, you quickly see the value of database migration files. The basic idea is that you define all of your database tables, columns, indexes, primary and foreign keys, restrictions and requirements in migrations files. These files can then be shared/transferred to other developers which then quickly and painless can generate the same database schema in their instance of the application.
When working with the database in Laravel I utilize the Eloquent ORM (Object Relational Mapping). Although I have written my own database classes in the past, the power of an ORM like Eloquent is to excellent to pass up. It’s a beautiful and simple ActiveRecord implementation for working with the database. Each table in the database has a corresponding “Model” class (the M in MVC) in your code which is used to interact with that table. The model is a powerful query builder towards the respective table; it also allows you to express relationships between the different models/tables for fluent and expressive interactions with the database in your code. Laravel does draw some inspiration from Rails. Visit this page to get some insight into the terminology above.
Laravel has a concept of a policy class, used to contain all authorization logic associated with a specific controller. This logic can then be invoked in the beginning in all controller methods that require authorization. Using Eloquent Models, it is super easy to check whether a user owns the resource she is trying to modify, which in many cases is all we need to verify in order to determine if the user should be allowed to proceed.
The Artisan Console is a command line tool that comes with Laravel to aid the development process. You can also write your own commands and register them with the framework. You can for instance auto generate the mock up for a controller or run a database migration. There is plenty of good info on how to create your own commands in the Laravel documentation. Companywathclist.com uses a command to check whether any Notifications are triggered, and if so sends an Email to notify the respective user.
Errors happen in all code bases, and if you don’t make it visible, you miss the chance of doing something about them. When an error happens in you code it should be captured and dealt with in a way that is as friendly as possible to your users. Laravel provides logging to files out of the box. You can easily set up the application so that daily log files are created.When it comes to exceptions; Laravel handles all exceptions with a Handler class. This class contains two methods: report and register. In the report method you can write any persistence logic, while in the register method is responsible for converting the given exception into an HTTP response to the browser.
Laravel provides a very simple and expressive API over the SwiftMailer library with drivers for SMTP, PHP’s mail function as well as many popular mail services like Mailgun and Sparkpost. Companywatchlist uses Mailgun to deliver Mail to its users. Which makes it easy to maintain mailing-lists and do batch deliveries to thousands of users using API calls, thereby avoiding bugging down the webserver. More on Mailgun in a bit.
User experience is key for the success of any application, and responsiveness is an important characteristic in respect to this. We all get frustrated when having to deal with slow sites. Just a few milliseconds can, when added up through long sessions, end up being a major frustration. Being able to set off time demanding tasks, like sending an email, in order to more quickly return a response to the user is what ques are for. Laravel provides several drivers for implementing queuing in your app. Companywatchlist uses Beanstalkd as its work queue. Beanstalkd was designed for the purpose of reducing latency in high volume web sites by running time consuming tasks asynchronously. The Laravel framework comes with a driver for Beanstalkd and you can take it in use simply by changing a configuration option.
On the topic of running code asynchronously I would like to quickly build some understanding of how the compiler reads through a PHP script. A PHP script is read line by line and there is no concept of running things asynchronously by branching tasks into their own threads. That means then you make an API call or send mail, the execution of your script is put on hold until the transmission of the data is completed. There are ways to get around this sequential execution of code. I really don’t know what I am talking about anymore, but promises and event loops is something I will look into in the near future.
My point is that queues allow you to get around the synchronous nature of the PHP language by deferring time consuming tasks to a later time. This way your users don’t have to wait for them to complete before getting served the desired page.
Tasks are often needed to run at a regular interval, like sending a weekly traffic report to site administrator, or do a nightly cleanup of the database. Laravel provides a central location where one can define all scheduled tasks for the application. This together with a fluent and expressive API makes defining scheduled tasks in Laravel a breeze. A result of this centralization is also that you only need to set up a single cron job for all you scheduled tasks, removing a lot of managerial overhead. Companywatchlist has a cron job being run every minute to check if any tasks are due, which is the case twice a day, once in the morning and once in the evening. This scheduled task runs an artisan command checking if any notifications are triggered and send emails if that is the case.
Read more about task scheduling in Laravel.
After reading pretty much all of the Laravel documentation, watched a bunch of Laracasts videos and read lots of articles, I still feel there are many holes in my understanding. Holes I truly believe will stay open until I have created a framework on my own and have dived deeper into the Laravel codebase. That will be a future project if I’m going to invest more time into programming and web development. The Laravel framework has much more to it than what I have mentioned here, if you want to read more about it head over to the Laravel Documentation.
The beauty of the web is that you can easily integrate third party services into your application, and take advantage of technology you could never make on your own. Companywatchlist relies on a couple of services to be able to deliver its functionality. They are briefly introduced below:
None of the functionality of Companywatchlist is made possible without the Yahoo Finance API. The data from Yahoo is not allowed to be used in commercial endeavors or even be redistributed. This is why I have locked access down to only the individuals I explicitly have allowed access. I have built an “Admin Panel” that gives me an overview of who have registered on the site, and I can from there either accept or deny their request for access.
I could get sued, but I very likely will not, because my footprint is so small and only a handful of people have access. No one actually uses the site, and it is clearly only for learning purposes. Which I even state on the middle of the front page. By sending a HTTP GET request compliant with yahoo finance’s AIP, you get back a CSV file containing the financial data on the requested company. CSV files are a pain to work with directly, so I parse the CSV into an array. From here I can pass the data to a view and have the blade templating engine generate the final result being returned to my users.
To make HTTP requests in your PHP code you can use Curl, streams or an HTTP client package. Companywatchlist uses the Guzzle HTTP client to handle HTTP requests and responses from within the application.
There are many different services you can use to get visibility and insight into you applications errors and exceptions. Sentry is a great service that does exactly this. By aggregating the exceptions that are thrown together with the requests that caused them into Sentry, you get great insight into how to improve your code or start trouble shooting problems; a very valuable resource indeed.
Mailgun is an email delivery service which makes it easy to maintain mailing-lists and do batch deliveries to thousands of users using API calls, thereby avoiding bugging down your webserver. You can do all sorts of fancy things; like define lists of routes to handle incoming emails. You can match on recipient and header values, and based on the matching route a corresponding action is performed, like forwarding the email to your application server or business email account. Mailgun provide great mechanisms to control how mail is sent and received by your domain.
The client side is comprised of several technologies working together to create the final user experience. The three core technologies of client side World Wide Web content products is HTML, CSS and JavaScript.
HTML (HyperText Markup Language) defines and describes the structure of webpages. HTML elements are the building blocks of HTML pages. The elements are represented by tags. These tags label pieces of content like “paragraph», «heading», «table» and so on. The browser does not display the HTML tags, but use them to render the content of the page.
CSS (Cascading Style Sheets) is a language that describes the style of an HMTL document. It describes how HTML elements should be displayed on the screen. CSS saves a lot of work because it can control the layout of multiple web pages all at once.
I don’t like CSS. This is probably due to the fact that I don’t know it well enough. I don’t find CSS intuitive with its different positioning and display types. Throw in some floating elements as well to make it extra confusing. The fact that everyone expects your page to look great on any device, from phones to 4k displays, also makes it hard to impress people with your CSS skills. Regardless, it is a very powerful tool that will allow you to create pretty much anything you can think of.
When first learning CSS you can get into the mindset that everything that has with how content is displayed, should be handled with CSS. But in fact, in many cases just using the innate properties of the different HTML elements, you can get the layout you want. And doing so with a lot less lines of code.
Companywatchlist uses Bulma, a CSS framework, to do most of the heavy lifting when it comes to styling. By following their documentation and applying pre built classes, your site will look great in no time.
JavaScript is one of the most popular programming languages in the world, which is largely due to it being the programming language of the web browser. JavaScript is standardized through the ECMAScript language specification. It is a high-level, dynamic, untyped, interperated run-time language. JavaScript is prototype-based, which is a style of object-oriented programming where behavior reuse (inheritance) is done via a process of reusing existing objects via delegation that serves as prototypes. The language support first-class functions, roughly meaning that functions can be passed as parameters to other functions. Some programming language theorists also require support for anonymous functions as well. In language with first order functions, the names of the functions are treated like any other variable with a function type. JavaScript support object-oriented, imperative and functional programming styles, making it quite versatile.
You don’t necessarily need JavaScript in your websites, HTML and CSS gets you a really long way, but if you need any sort of custom logic on the client side, JavaScript is the answer. Example use cases could be form validation, dynamic updates of HTML/CSS or performing AJAX (Asynchronous JavaScript and XML) requests.
The client side framework: Vue
Now with some insights into what a server side framework provides, what exactly are the problems a JavaScript framework solves? In a broad perspective Vue tries to solve the same problems like any other framework, like how to structure code and ideas, how to solve common issues any project must address and further to provide resources to make the development experience easier, faster and more pleasant.
The thing that stands out most to me is that JavaScript frameworks makes it a lot easier to manage updates of the view (HTML and CSS) depending on the state of the application. As data flows through your application and the data/state changes, this needs to be reflected in what the user sees on the screen. By having the framework handle the nitty gritty details of how these dynamic updates are performed by binding the data to the view, the life of the JavaScript developer is made a lot easier.
Vue is designed to be incrementally adoptable. The core of the framework has a very small footprint and is mainly concerned with the view layer. You can easily integrate Vue with other libraries or existing projects. Vue is also perfectly capable of handling big single page applications when used in combination with modern tooling and supporting libraries.
There are many other JS frameworks out there. You have Angular (maintained by Google) and React (maintained by Facebook), as the biggest players in the space. Vue is the new kid on the block, but certainly has a lot of traction and the future looks bright.
Web development imposes many challenges on developers, and to handle the complexity of development, deployment, hosting and maintenance, many useful tools have been built. In the following sections I will mention the tools and services I have taken advantage of during the creation of Companywatchlist.com.
Version control is an absolute drag to do on your own. You add a new feature, but it did not work out as well as you thought it would, so you need to revert back to an earlier state, but you have made changes all over the place and it’s difficult to do. Or, you are collaborating with other developers on a project and all the different contributions must be managed somehow, you can’t just have ten developers add code all over the place with no gatekeeping or central control.
Git is a version control system helping you manage your code under its development and life. Instead of me trying to explain its features I’ll refer you to git-scm.com.
Github is a hosting service for Git repositories. On Github you can collaborate with other developers on your favorite projects, share your own projects with the world, or simply store your code in the cloud. Github also has a lot of other neat little features. Check em out at github.com/features.
Forge is a server management system developed for PHP. Forge help you provision and deploy your PHP application to a number of cloud platforms. Forge installs software, keeps servers up to date, synchronizes the server with the applications Git repository ( push to deploy), installs and manages SSL certificates and helps you manage your queue workers.
Deploying and configuring a web server form scratch was not really a goal for me during this projects, I had plenty of challenges just writing the application, so I decided to outsource this problem to Forge.
You need somewhere to host your server, doing this yourself is something you do not want to do. There are many great could platforms that allow you to spin up and manage virtual machines. AWS is a feature rich cloud service provider. You get one year of “free tier” subscription where you can basically host a t2.micro (1 virtual CPU, 1GB ram, 20GB HDD) for one year for free, plus some other stuff. I also signed up for their “student program” and got $150 of credit to use, but it expires one year after signup.
The AWS Cloud provides a broad set of infrastructure services, such as computing power, storage options, networking and databases that are delivered as a utility: on-demand, available in seconds, with pay-as-you-go pricing. AWS also deliver a whole ecosystem of services to build massive distributed, scalable and high performing web applications. Today over 90 services are available in the AWS could. A great place to start learning is the AWS whitepapers.
I used GoDaddy to purchase the “Companywatchlist.com” domain. It cost me about 1$ for one year. I mainly decided to go with GoDaddy because of the low prices. All domain related configurations needed to set up name resolution and email was fairly easy and accessible through their GUI.
As I mentioned earlier the adoption rate of new technologies in different web browsers differ and the latest and greatest technologies that we want to start to use right away are often a long way from full adoption. The JavaScript standard that is undergoing implementation today is ECMAScript 6 (ES6). Although we wish we could just write ES6 and be done with it, we can’t, because our code will then not work in a big portion of our users web browsers. The remedy to this compliance issue is to translate ES6 to the previous standard ES5. Another issue we run into when writing JavaScript applications is that when we split out code into multiple files we need to put them back together before sending it to the browsers. These issues are addressed by Webpack, doing both translations from ES5 to ES6 (+ SASS to CSS) and bundles all your JavaScript (and CSS) assets together.
These Webpack tasks can be managed using Gulp, a tool to automate your build processes. Laravel uses gulp to automate the build process of Laravel applications, but the framework also comes with a super simple interface towards Gulp called Elixir. Writing Gulp files/configurations can be cumbersome if you have never done it before. Elixir enables you to leverage the power of Gulp without you having to know much about what happens underneath the surface.
Companywatchlist’s client side code is written using ES6 and SASS and therefor uses the technologies mentioned above.
There are many more things I could have put into this article, but all major topics regarding the development of Companywatchlist.com has been covered. This is a project I could take further in the future, and I might do so, but at this point it time I will not continue to develop the site.
First of all it has been a great learning experience to build the site. All the challenges I have had to overcome have greatly increased my confidence and technical ability. It has also spiked my imagination for what might be possible for me to do in the future. Other than the obvious knowledge i have gained in PHP, JS, HTML, CSS and general web development, I have benefitted in a number of other ways to. Compared to before I started learning web development and building companywathclist.com I feel that I have gotten better at general problem solving, systemizing, abstract thought, project management and self-discipline. Another side effect of getting into programming and web development has been a much deeper understanding and appreciation of information technology and computer science in general. I feel more competent and confident when approaching technical problems of any kind now than what I did before. Despite the massive amount of time I have spent on this the past year, and the not infrequent feeling of frustration or being overwhelmed, I certainly feel it was worth it.