Beginner’s Guide To CI/CD Pipeline From Scratch

If you want to make your software more efficient and easier to alter, it’s in your best interest to set up a ci cd pipeline. A pipeline will make your operations so much more efficient!

Of course, if you have never built a CI/CD pipeline before, it can seem like a lot to take in. It can be overwhelming trying to wrap your mind around all of the things that you need to do to create a successful CI/CD Pipeline. Thankfully, we’re here to help. Here’s how you build the ideal CI/CD pipeline from scratch.

What Is a CI/CD Pipeline?

If you’re new to the world of DevOps and software development, you may be wondering what exactly a CI/CD Pipeline is. In essence, it stands for Continuous Integration/Continuous Delivery. It is pretty vital to DevOps. Having one of these pipelines to use means that you can change code much more easily, and then the code is tested and used for deployment and delivery. The result of using a CI/CD pipeline is often that your code is significantly faster than you could ever have dreamed.

Let’s break this down a little. You’re creating a piece of software. This begins with someone writing the code for the software. The code is put into something like svn – basically a version control system by the developers. Then the build phase is undertaken, which is where the pipeline ultimately starts its first phase. Then you enter the version control phase, and it may then go back to the building phase if there are any issues.

After this you’re then onto the testing phase, and then it’s time for the deploy phase where it will go to the test server so you can see how it works. It goes to auto test then, and finally onto the deployment to production phase. This lifecycle can be repeated as many times as necessary until the developers are happy with the final product.

With a well done CI/ CD pipeline, you can automate the above process, which makes building and deploying the software much easier. It also means much more efficiency and less downtime overall. 

The Benefits of CI/CD Pipelines

There are a number of benefits to CI/CD pipelines. For starters, it increases efficiency. If you use CI/CD technology, you can deploy software applications much faster than you would otherwise. This leads to better output and often improved results.

Likewise, using this technology also means that if you are developing software in your own company, you won’t have to spend lots of time doing manual tasks. Instead, you or your staff can focus on doing more important work. A CI/CD pipeline will allow you to feel more confident that any errors are caught straight away too, rather than finding out later down the line, which can result in a lot of wasted time. 

Having a CI/CD pipeline also gives you the opportunity to play around with code. Because the pipeline will often pick up any errors you are making, you can have the confidence to experiment with new coding with far less risk than ever before. You can maintain the software much more easily after launch too, since you can find the errors and issues without any trouble.

As a whole, using a CI/CD pipeline also means that your code is likely to be of a higher quality too. The pipeline will help you to ensure that any errors are immediately rectified and pointed out, minimizing expensive and time consuming mistakes. 

Creating a CI/CD Pipeline From Scratch

The honest answer is that there’s no right or wrong way to set up a CI/CD pipeline. Different people will have different methods. Likewise, there’s also a lot of different technology out there that can allow you to create a pipeline, so the method will vary depending on the technology that you use. For the purpose of this article though, there are some basic pieces of software that you will need to have access to: Python and Docker. It will also help if you have an account on GitHub, Travis CI, Heroku and Docker Hub. 

Here are some of the basic steps.

Open Python and Start Writing the Application

For the sake of simplicity, let’s start out by creating a web application. This is going to be our guinea pig for the process. We can begin by making a buzz generator program.

To start, make a whole new directory and give it a name. Then when you are in the directory, create another one inside of it, perhaps under the name ‘buzz.’ You will need to find the code for this to put into the system. You will then have created a little bit of code – make sure that you save it into a file called generator.py. You’ll need to create another file in the directory called __init__.py for this code too. 

Your structure for the project should look roughly like the below:

cicd-buzz/

buzz/

__init__.py

generator.py

When you have done this, run the script. 

Create Automated Tests

When it comes to your CI/CD pipeline, it’s essential that you have automated tests in the mix otherwise it’s not going to work. 

You’re probably getting sick of creating directories right now, but you’ll need another one – name this ‘tests’ and put it in the root of the directory. You will need to import some code into the system in order to create the automation, and when you’re done you can name the file – something like test_generator.py should work just fine.

Move onto the ‘pytest’ framework in order to test the software – you will need to install it using ‘virtualenv’ and you will then need to execute a command: [cicd-buzz] $ virtualenv venv.

You’ll then have a new directory, and you’ll need to create a requirements.txt file that will display the pytest.

Use GitHub 

Open your GitHub account and then create a public repository – this should be called something like cicd-buzz, or whatever other name makes sense depending on the CI/CD you’re creating. 

You should be able to open the project directory then and create a file – this will be called ‘.gitignore’ and should have one line – venv. Then initialize git and put the code into GitHub.

You may need to do some troubleshooting if there are any issues with access rights on the system.

Open Travis CI and Run the Tests

You should then open Travis CI – a continuous integration service to help you with creating your pipeline. It’ll work with GitHub, and all you need to do is sign into it with the login details from your GitHub account.

Allow the software to begin the build – you will just need to turn the switch on for the software using the Github repository for the project. The final thing to do is to add a file called ‘.travis.yml’ into the project directory’s root. Put this file into get and then push the alterations that you have made. 

The code changes will start to show up on Travis CI after a little while.

Put Better Code Hub In The Pipeline

The pipeline is now in place and should check how the code is working, thanks to the automated tests that we set up earlier on. The next step is to add something called Better Code Hub in the pipeline – this will monitor the quality of the code so that you can be sure it’ll be of a good quality later down the line. Log into Better Code hub using your details for GitHub, then access your repository. Press play, and click ‘go’ on the prompt from Better Code Hub to activate the analysis.

Use Your Code to Create a Web App

You’ve done a lot of the heavy lifting now – nice! Now you need to turn it into an app. You’ll need to add a little bit more Python Flask code to your pipeline in order to do this. Add the code, and then put it into a file entitled ‘app.py’ – put this into the root project directory. Run the web app, add the commit and see your handiwork!

Open Docker to Containerize

You’re going to need to open Docker now and make a self contained unit of the app ready to deploy. This may seem like a lot of effort, but it’ll ultimately be to your benefit later on if the code in question is in a self contained unit that’s small. 

Put some code into docker in a file you will create called Dockerfile – this should be in the project directory’s root. Then commit and push the changes.

Deploy on Docker Hub

Then you will need to deploy the code so far to Docker Hub onto a central image registry. Sign into docker to do this and add the right deploy code to a directory you will call ‘.travis’ in the project directory. You will need to work both on Travis and on Docker Hub for this step.

Heroku Deployment

When that’s done, you can then deploy on heroku. Sign into it and install the command-line toolbelt from Heroku to execute the right commands using more code. You can then get onto the app. Automate the deployment process from here by adding code to a file ‘deploy_heroku.sh’ in the ‘.travis’ directory we mentioned earlier. 

Then put two additional environment variables on Travis CI – look for the Heroku API key. You should be able to find this on account settings on Heroku. You’ll find a ‘Heroku Create’ command there. Push all the changes on GitHub and you’ll find a Docker image on Docker Hub and Heroku when you have successfully completed all of the steps.

Finished! 

Now your CI/CD pipeline is all ready to go. Play around with your workflow as you see fit – any changes that need to be made should be much easier to do now. You can enjoy coding to your heart’s content without messing around with tiny errors in the process. Coding has never been easier.

Conclusion

A CI/CD pipeline is useful, but you need to make sure that you execute it correctly. Hopefully now you too know how to create the ideal pipeline for your new piece of software.