Migrating/Updating my TinyDB MAC API Introduction

On By dustin

Since setting up my home lab five years ago, I’ve loved to track lots of data points about my network. This all initially started with huge CSV files that I would rotate either yearly or monthly and display with a graphing tool called dygraphs. As I continued to learn, I slowly introduced projects on my network that aimed to use more up to date options such as Grafana, FastAPI, MySQL, etc. This blog series is going to cover a project that will migrate one of my services that was created somewhere in the middle of my learning journey, to instead use more modern structure.

The original project was inspired by this Cisco Devnet Snack Minute:

In the video, Kareem introduced a simple database option, called TinyDB, and used this to create an API. At that moment, I decided I would look to migrate one of my many CSV files into a TinyDB project. Following Kareem’s instructions, and then tailoring some small details for my own project, I was able to create a very simple “API” that provided me information on scans against my network, vpn use, backup statuses, individual service statuses, as well as a list of what MAC addresses where present on my network:

It was a bit basic…but an exciting to get a new service up and running. Several of my other services continue to query this API even today. While the API has been up and running for two years now, I have decided that I need to migrate off the TinyDB option because I am facing a similar problem as the CSV files (size!) and am looking for more experiences working with MySQL. I have broken this project down into four primary chunks:

  1. MySQL Portion: focuses on establishing the database structure that will support my project.
  2. SQLAlchemy: focuses on establishing SQLAlchemy functions, and other supporting python, to interact with my database.
  3. FastAPI: focuses on getting FastAPI up and running.
  4. Portability (Docker?): focuses on providing an easy to deploy solution.

This project will primarily focus on migrating the MAC address portion of the above solution:

I’ve used FastAPI in a few other projects, and am excited to see what this will look like in a few months.