Summer 2024, CSCI-GA.2820-001, Graduate Division, Computer Science


DevOps and Agile Methodologies



Instructor

John Rofrano
- Senior Technical Staff Member, IBM Research, T.J. Watson Center [intructor profile]

Lectures and Labs

Wednesdays 5:10PM-7:30PM
Wednesdays 7:40PM-8:40PM
ONLINE (via NYU Zoom)

Course Abstract

DevOps is more than just a fusion of development and operations. It is the cultural change that embraces Agile methodology, Lean Manufacturing, Social Coding, Behavior Driven and Test Driven Development, with new tools to provide Continuous Integration, Testing, and Delivery, and a fundamental shift in what it means to "move into production". Developers are empowered to "fail fast" and deploy early and often. Cloud Computing is at the heart of this movement because it allows teams to move at the "speed of need". Cloud Native applications built from loosely coupled microservices require DevOps automation to manage the often complex production pipelines. DevOps allows businesses to move with increased agility and stability so that they can "innovate at scale".

Equally as important is understanding what DevOps is NOT:

  • DevOps is NOT about System Administration. It is about developers and operators working together, preferably on the same team, optimally being the same person!
  • DevOps is NOT a Job Title. It is unfortunate that some companies who don't understand DevOps have renamed the System Administrator job as "DevOps Engineer". If you are not doing development, then you're just a operator regardless of your fancy new job title.
  • DevOps is NOT a separate Team. The whole purpose of DevOps is break down siloed teams. Creating a new team called DevOps is an anti-pattern which only proves that you don't know what DevOps is.
  • DevOps is NOT one size fits all. There is no DevOps manifesto specifically for this purpose. DevOps is an evolving culture driven concept and each semester this course is updated to reflect the latest thinking.

This course is an Advanced Software Engineering course but it is more than just learning about DevOps practices. It is an emersive experience into the the culture of DevOps. Students will work in small Agile teams of 4 to 5, to plan their work and build a cloud native microservice using Python and Flask that they will deploy to the IBM Cloud. They will join a Slack team for the duration of the course where they will collaborate with their instructor and team members in their own channel for their project just like developers do in the industry and on Open Source projects. Upon completing this course students will know from hands-on experience how to work as an Agile team, socially code, develop microservices with RESTful API's, setup an automated DevOps pipeline to continuously integrate and test their code changes and deploy the Cloud Native applications that they develop, onto the Cloud. Students will be given access to IBM Cloud and Watson Services for the duration of the course and will be submitting their homework on GitHub and deploying their homework to the cloud for grading.

This course is 50% lecture and 50% hands-on so come prepared to learn by doing.

Prerequisites

Prerequisite knowledge for this class is CSCI-UA.0002-008 Introduction to Computer Programming in Python or equivalent. Programming in either Python, Ruby, Java, or other modern programming language is required. The course will be taught using Python 3.7 with Flask and other libraries and test tools like PyLint, PyUnit, Nose, and Behave. We will use Object Oriented Programming techniques to represent resources in our REST API so an understanding of OOPS is helpful. All of your work will be stored on GitHub so a working knowledge of git would be helpful but not necessary as I will teach you what you need to know in the hands-on labs. Finally, it would be helpful to have Unix skills because the course will be taught on Ubuntu Linux and understanding basic Unix file manipulation commands (e.g., ls, rm, cat) will be required. This is a technical hands-on course. If you are not technical, this is not the course for you but if you want to aquire a broad set of modern software engineering skills, you've found to the right place. ;-)

Since this is a hands-on course, students will need to have a laptop in class with at least a Core i5 CPU and 4GB of memory. A Core i7 with 8GB of memory is recommended. Students with an Apple M1/M2 Silicon Mac will be fine. We will use Visual Studeio Code and Docker Desktop throughout the class labs and instructions on how to install these will be given the first day of class. A Mac is preferred but Windows computers can be used although additional software will be needed to be installed as Windows does not have a bash shell, or git client, or other unix commands (e.g, ssh, scp) needed to work on remote cloud servers. A Linux laptop will also be fine.

Topics Covered

  • DevOps Introduction, History, and Overview
  • Social Coding with GitHub, Slack, & Webhooks integrations
  • Agile Development and Planning using ZenHub
  • Review of the Spotify Engineering Culture
  • The Twelve-Factor Application (12factor.net)
  • Using and creating Infrastructure as Code and Software as a Service
  • Automating Development Environments with Vagrant, VirtualBox, and Docker
  • Creating Cloud Native RESTful microservices with Python Flask
  • Introduction to Platform as a Service using Kubernetes
  • Automating Testing with Behavior Driven Development and Test Driven Development
  • Creating a DevOps CI/CD Pipeline with tools like Travis CI, and Jenkins
  • Introduction to Docker and Kubernetes Container Clouds
  • Overview of Automated Deployments using Chef, Puppet, Salt, and Ansible
  • Security and DevOps

Grading:

In this course you will work as part of an Agile Development Team. The project assignments will progressively build a 12 Factor compliant Cloud Native RESTful microservice that you will design, develop, and deploy into both Cloud Foundry and Docker/Kubernetes Container Cloud using the DevOps methods taught in the classroom. While you will work on the project as a team, students will also be evaluated based on their individual contributions.

40% Project Assignments
30% Mid-term Exam
30% Final Exam

Academic Integrity

Please review the departmental academic integrity policy. In this course, you will work as a team and should discuss assignments with other students on your team but the work you turn in must be your own. You must do the projects as a group but not with other groups and without consulting previous years' students, code, etc. You should help other students and groups on specific technical issues but you must acknowledge such interactions. Copying code or other work without giving appropriate acknowledgment is a serious offense with consequences ranging from no credit to potential expulsion.

Feedback

I recognize that every student has a different level of background knowledge and prior experience when it comes to technology. While this course is an advanced software engineering level class and the ability to program in Python is required, an abundance of software examples will be given and topics will be presented assuming very little prior exposure to the topics. With that said, every student learns differently and I want to ensure that each of you is getting the maximum amount out of the course content as possible. Please do not hesitate to get in touch with me via e-mail, or direct message me on Slack if you feel as though you are falling behind or you are not understanding a certain concept.

Related references and quotes:

DevOps Digest Market demand for DevOps skills is growing, and DevOps engineers are among the highest paid IT practitioners today, according to the DevOps Salary Report, based on salary data gathered from Puppet Labs' industry-recognized State of DevOps Report.

Tech Beacon: The data confirms the strong demand for DevOps professionals as a growing number of organizations begin implementing the development model in their organizations. "We've seen DevOps spread beyond [web] companies and bleeding-edge enterprises to a more mainstream enterprise audience that includes almost all major verticals."

New York Times: As some tech sectors show signs of slowing, cloud services have created remarkable demand for highly educated engineers and mathematicians. And they are being compensated very well.