Simple JSON API with Python and Flask

This should be the first of a few blog posts explaining how to get a simple API up and running with Python, and using the Flask framework. I’ll try and link all these together once I get more up. And if you’d like the source code at any time, you can grab it here: github.com/billturner/simple_flask_api. The branch for this part will be01-bare-bones.

I’ll be using Python 3.x, and I’m going to assume you have it all set up and in your path. If you don’t have it set up, you can find a downloadable installer on the Python web site.

Setting up the virtual environment and installing Flask

We want to create a directory for our Python files, so make one of your own choosing; I used simple_flask_api. Change into that directory, and from here on out all files should go in here. I’ll use the virtual environment ability built into newer versions of Python, and I’ll call the environment venv:

python -m venv venv

To enter the virtual environment, you’ll need to run the appropriate initialization script. Since I’m on Windows, I’ll be executing this command in PowerShell, so what I use may be different than you. For most Linux and OS X users, you’ll use another command which I’ll list below (for other platforms, there are instructions in the Python docs):

# for Windows/PowerShell users, you should run:
.\venv\Scripts\Activate.ps1
# for most Linux and OS X users, you should run:
source ./venv/bin/activate

Now, you should have a prompt that has (venv) at the beginning, like this (the ... is just me shortening the path):

(venv) PS C:\Users\...\simple_flask_api>

For the rest of this tutorial, we’ll assume you’re in this directory and have activated the virtual environmen. Once you have activated the virtual environment, you can install the first of the two libraries we’ll need for this part of the tutorial, Flask:

pip install flask

You should see some text scroll by on the screen as the library (and its dependents) are installed. Now we’re ready to build an API.

Hello world

A tutorial isn’t a tutorial without a “hello world” example, so let’s write up a quick one (a version of which is in a lot of Flask tutorials). Create a new file named app.py and place the following code in it:

from flask import Flask

app = Flask(__name__)

@app.route("/", methods=['GET'])
def hello():
    return 'Hello world!'

if __name__ == '__main__':
    app.run(debug=True)

What we’re doing here is importing from the Flask library, initializing the app with Flask, and then creating a simple route, and a function to be called when that route is called. For this tutorial, we are only going to be fetching data from our API, so the only HTTP method we’ll need is GET.

We’re adding debug=True to app.run() so that when you make changes to the source code, it will automatically reload the app. Now, to start the app running, execute this command where we’ve been building this:

python app.py

You should get some feedback from Flask starting up, and then a URL you can use to access the app in the browser. Should be something like http://127.0.0.1:5000/. If all went well, then you should see the string “Hello world!” in your browser.

But wait, we’re building an API, and a basic string isn’t helpful when building an API, JSON is. So, let’s also import the jsonify utility from Flask and then return a JSON object from our hello() function (I’m only showing the changed lines here, and the [...] signifies code that doesn’t change):

from flask import Flask, jsonify
[...]
@app.route("/", methods=['GET'])
def hello():
    return jsonify({'message': 'Hello world!'})
[...]

With that change, and reloading your browser, you should have a JSON object response, instead of a string:

{
  "message": "hello world!"
}

We now have a JSON API! It doesn’t do much at the moment, but it’s a start.

Adding mock data

Create a file named mock_data.py and place the contents of this gist into it. Since the end goal of teaching myself Flask is so that I can get Lists of Bests up and running again, we’ll use some fake data for possible book and album lists for the site. We’re including it in a separate file just to keep app.py as simple as possible.

At the top of app.py, we’ll pull the mock data into our app:

from flask import Flask
from mock_data import lists

And now that we have the lists, let’s add a new route to handle displaying the lists. Add the following after our hello() function:

[...]
@app.route("/lists", methods=['GET'])
def get_lists():
    return jsonify({'lists': lists})
[...]

Once the app reloads, if we now try to access http://localhost:5000/lists in the browser, we should get a JSON representation of the lists from our mock data file, that looks like this:

{
  "lists": [
    {
      "description": "This is  the description for the first list",
      "id": 1,
      "name": "This is an example list",
      "type": "albums"
    },
    {
      "description": "A description for this list should be different",
      "id": 2,
      "name": "Another list",
      "type": "books"
    }
  ]
}

Getting a specific list by id

In order to get lists by a passed in id, we will need a new route and handling function. Let’s add this below our last one for all the lists:

[...]
@app.route("/lists/<int:list_id>", methods=['GET'])
def get_list(list_id):
    _list = [_list for _list in lists if _list['id'] == list_id][0]
    return jsonify({'list': _list})
[...]

I’m using _list for the variable name above, since list is a reserved word and can cause problems. And since we’re only getting a single list, I’m grabbing the first of the array with [0]. This is brittle, but will work for our simple case where we know exactly what the data looks like.

Once saved, you can now access this by passing a valid id to the URL as the last part of the URL, like http://localhost:5000/lists/1.

We now have a list, but where are the list items that we added to the mock data? Let’s make sure we’re including the list_items in our import statement, and adjust our function just a bit and add the proper list items into the response:

[...]
from mock_data import lists, list_items
[...]
@app.route("/lists/<int:list_id>", methods=['GET'])
def get_list(list_id):
    _list = [_list for _list in lists if _list['id'] == list_id][0]
    _items = [_item for _item in list_items if _item['list_id'] == list_id]
    if len(_items) > 0:
        _list['list_items'] = _items
    return jsonify({'list': _list})
[...]

After saving and reloading, the output should look like this:

{
  "list": {
    "description": "This is  the description for the first list",
    "id": 1,
    "list_items": [
      {
        "creator": "Radiohead",
        "id": 1,
        "list_id": 1,
        "title": "OK Computer"
      },
      {
        "creator": "Miles Davis",
        "id": 2,
        "list_id": 1,
        "title": "Kind of Blue"
      },
      {
        "creator": "Thelonius Monk",
        "id": 3,
        "list_id": 1,
        "title": "Monk Alone"
      }
    ],
    "name": "This is an example list",
    "type": "albums"
  }
}

Nice! Now we have a list and all its items. We could probably clean things up a bit - like removing the list_id attribute from the items - but this works just fine for our basic example.

Testing

Since we’re good little developers, it would probably be a good idea to write some tests to validate that the API is working as expected. We’ll be using pytest as the library, so let’s install that:

pip install pytest

The Flask documentation has some examples of testing using pytest, so we’ll use those as reference while making our own. To start, create a file named test_app.py in the same directory as the other files. First, we’ll need to import pytest and our app:

import pytest
from app import app

In order to reduce a bit of extra typing, let’s write a test fixture for the app client used for testing:

[...]
@pytest.fixture
def client():
    return app.test_client()

We’re not saving a lot of typing here, but later if you have to initialize a database, or set up anything else, this fixture would be a good place to do that. First, let’s write a super-simple test - that makes use of our client fixture - to make sure we’re getting the right kind of HTTP response code, and MIME type back from our JSON API server:

[...]
def test_json_with_proper_mimetype(client):
    response = client.get('/')
    assert response.status_code == 200
    assert response.content_type == 'application/json'

Now, in order to run this our first test, run the pytest command at the console:

pytest

And you should get some output that looks like this (minus a bunch of long lines like ===========):

platform win32 -- Python 3.7.1, pytest-4.4.0, py-1.8.0, pluggy-0.9.0
rootdir: C:\Users\...\simple_flask_api
collected 1 item

test_app.py .                                [100%]

1 passed in 0.18 seconds

The little dot next to test_app.py should be green, signifying its passing. You’ll get a green dot for each passing test, and if you get an error, it will give you a wordy explanation telling you what went wrong.

With our first test passing, let’s add one where we test the JSON in the response. Here’s one that checks that the JSON from our root URL is correct - and we’ll use the get_json() helper method to parse the JSON for us:

[...]
def test_hello_world(client):
    response = client.get('/')
    json = response.get_json()
    assert json['message'] == 'Hello world!'

If you run pytest again, you should now have 2 green dots. To finish this part up, let’s test that we’re getting the right number lists from our /lists API endpoint, and that the endpoint for a single list also returns the list items:

[...]
def test_lists(client):
    response = client.get('/lists')
    json = response.get_json()
    assert len(json['lists']) == 2

def test_single_list(client):
    response = client.get('/lists/1')
    json = response.get_json()
    assert json['list']['name'] == 'This is an example list'

def test_single_list_items(client):
    response = client.get('/lists/1')
    json = response.get_json()
    assert len(json['list']['list_items']) == 3

Running pytest now should show that we have 5 successful tests.

Catching errors

Right now, if you try to access any URL outside of the ones we’ve specified, you get Flask’s generic 404 HTML error page. It would be nice to customize this a bit to return a JSON error since this is an API service and not a regular web site.

First, let’s write an errorhandler for 404 errors to display them as JSON. In app.py, you can place this before the if __name__ == '__main__' line:

[...]
@app.errorhandler(404)
def not_found(error):
    return jsonify({'error': '404 Not Found'}), 404
[...]

Now, whenever Flask would normally handle a 404 error, instead of returning its built in generic HTML page, it will now return this JSON response. If you try and load a non-existent URL - like http://localhost:5000/fakeurl in our app, you should now get our JSON error.

{
  "error": "404 Not Found"
}

And now that we have a test file all set up, let’s write one to validate what we’re seeing in the browser. You can add this test below all the others:

def test_not_found(client):
    response = client.get('/not/real/url')
    json = response.get_json()
    assert response.status_code == 404
    assert json['error'] == '404 Not Found'

And that should pass like all the other tests. Finally, there is one more case I’d like to test: what if you try and access a list that doesn’t exist. We know we have lists with id of 1 and 2, but what happens if we try for list 3 with http://localhost:5000/lists/3? If you do try it out, you’ll get a nasty Flask error message that says IndexError: list index out of range at the top. We should account for lists not being available.

We can use the built-in abort() method to trigger a not found error if we try and access a list that doesn’t exist. First, we’ll add abort to our import statement, and then update our get_list() function to trigger this when we can’t find the list:

from flask import Flask, jsonify, abort
[...]
def get_list(list_id):
    _list = {}
    search = [_list for _list in lists if _list['id'] == list_id]
    if len(search) > 0:
        _list = search[0]
    else:
        abort(404)
    _items = [_item for _item in list_items if _item['list_id'] == list_id]
    if len(_items) > 0:
        _list['list_items'] = _items
    return jsonify({'list': _list})
[...]

We had to add a bit more code to capture the case where we don’t find a list based on our data, but not too much. Now, if you try accessing a list that doesn’t exist, then you’ll get that handy 404 error we created above. And like before, let’s add a test to validate the changes. Again, just add to the bottom of the test_app.py file:

def test_list_not_found(client):
    response = client.get('/lists/99')
    json = response.get_json()
    assert response.status_code == 404
    assert json['error'] == '404 Not Found'

When you run pytest now, you’ll see a new green dot for a passing test, but also this helps show that the adjustment of our get_list() function still works as expected.

Wrapping up

Even in building our very basic JSON API, it does take a bit of set up to get things started. And at this point we’re not even accessing a database, or allowing updates to the API with POST and PUT requests. However, something like this tutorial was what I initially looked for; I just wanted the absolute basics. And this basic tutorial gives me a good base to build more complex features upon.

And it goes without saying that some of the code above may not be the best, but I’m sure I’ll learn to write more idiomatic Python as I continue.

I think that’s about it for this part of the tutorial. For the next one, I’ll likely pick one of the API libraries - RESTful or RESTPlus - and see how much of a difference they make.

If you have any feedback or questions about this tutorial, please let me know.

Resources

Some resources that helped out in this first part of the tutorial (in addition to the documentation already linked above):

~ ~ ~

How to get started

A wonderful comic by Alex Norris on learning distractions

With the internet, there are so many resources available to help you learn about something new - a new recipe, a new hotel, a new celebrity couple, and a new-to-you programming language. In fact, there are often too many resources and opinions out there, and finding the right place to start learning can be quite daunting. I’ve found this to be the case lately while trying to learn Python.

In addition to the book and tutorial I mentioned last post, I’ve also been going through another tutorial on how to write a roguelike game using Python. Despite the varied resources, I feel like I’m beginning to get a better grip on the language and its syntax.

But then I wanted to start working on an initial Flask API for Lists of Bests. Sounds somewhat simple enough, but once I started looking at documentation and tutorials, I quickly became overwhelmed. Flask by itself could probably do what I want, but I found references to all kinds of libraries to supposedly make things easier. There’s RESTful, and then RESTPlus, and another with the awesome name Marshmallow. And I’m sure that’s not all of them.

I think I may need to take another approach at this. Since I’m still quite new to the language, I think I’ll start with the most simple example first - without any extra libraries or distractions. I think if I can get my head around the basics, it will be much easier to add in those libraries later. And I may not even need them. I’ll attempt to document my learning into a couple of blog posts in the chance that someone else happens to have fallen into the library well like I did.

The excellent and oh-so-true comic by Alex Norris (@dorrismccomics).

~ ~ ~

Learning Python

When I initially thought about how I’d rebuild Lists of Bests, I had in mind to use some technologies that were somewhat new to me, so I’d have a chance to learn along the way. Well, since that post those new technologies aren’t all that new to me anymore. At my new(-ish) position at Arcadia, I’ve been writing quite a bit of React along with Ruby on Rails (and its API capabilities). So, there isn’t that much of a need to learn those tools from scratch.

So, what to use instead? Well, I’m thinking it may be Python and the latest version of Angular (I still work some in AngularJS 1.x, but not at all with any version greater than 1.6). Why these? Well, I think the primary goal of getting the app up and running again was to use the process as a learning opportunity. If I’m using familiar technologies that I’ve used before, or currently during my day job, then it won’t be as much of a learning experience.

I’ve kick-starting my learning with the ebook version of Python Crash Course, and along side that, the Flask Mega-Tutorial. The book is doing a fantastic job of teaching me the basics, and the Flask tutorial is helping me get a handle on pieces of the web programming side. The Python Crash Course book does include a Django section which I haven’t encountered yet, but I think Flask may be more along the lines of what I would use for Lists of Bests.

It was great to see how Pete (aka RasterWeb) is starting to learn Python as well. In his post, he mentions how “learning a new (computer) language is probably easier now than it was 25 years ago” and I totally agree. Way back when - like with Pete, I was probably learning Perl - the resources available were much more limited than they are now. Just getting your system set up to run a particular language could take a long time with lots of trial and error. Now, with tools like Homebrew on a Mac, and Chocolatey on Windows, getting a programming environment up and running only takes a few minutes.

Making this change may make it take a bit longer to get the old site up and running, but I think it’ll be worth it. I’m excited to actually get it started soon.

~ ~ ~

Back to Windows?

Crappy photo of the MSi laptop I bought

About a year ago, I bought a Windows laptop so that I could have a non-work computer to use. I thought about getting another Mac (for over 10 years, have been using them for work and home), but I didn’t feel like shelling out how much they’re asking for the newer MacBook Pros. So I started looking at Windows laptops for their power, but also the ability to play games that aren’t available on Macs. I bought an MSi laptop, and it’s mostly succeeded as becoming a fill in for the Macs I’m used to.

However, getting things exactly like I’m used to has been a bit of a struggle. I was happily surprised at the software ecosystem for Windows, but still there were a couple of hiccups. Some applications and tools can be quickly installed with the Chocolately package manager. Thankfully, many of the other tools I normally use have a Windows version, like Visual Studio Code, 1Password, SimpleNote, and others.

The one thing that’s not quite set up how I would like it is the more Unix-y tools I’m used to, like Vim, Ruby, Node.js, Git, ssh, and others. There are some packages available from Chocolately for these, but PowerShell isn’t bash, so getting all these tools working together how I’m used to is still a work in progress.

There are also some neat new features in Windows, like the Windows Subsystem for Linux, that I’m eager to play around with a bit more. Hopefully, details about things like that will end up as blog posts as well.

2019-01-14 — #programming #windows
~ ~ ~

Making my own sketchbooks

Some of the sketchbooks that I have made so far

Since I’ve been sketching more the last few years, it sometimes seems like I’m always searching for the next sketchbook to buy. I’ve been a huge fan of the different sizes of the Stillman & Birn Alpha sketchbooks, but I thought it could be fun to make some sketchbooks of my own.

When I started looking for some bookbinding tutorials, I came across Sea Lemon’s videos on YouTube, and specifically her simple saddle stitch bookbinding tutorial. That definitely seemed like something I could pull off. Plus, it didn’t seem like the supplies for those kinds of books would set me back too much.

I visited the local Dick Blick store and picked up a few supplies, like a few sheets of thick posterboard in various colors (for book covers) and a big roll of cheap-ish drawing paper for around $30. The paper wasn’t all that I needed, though. All that was left were a few more essential tools (like a awl, a bone folder, and some thread), and I could make my own sketchbooks with relative ease.

The first couple haven’t been the best quality, and they certainly don’t have perfectly square corners, but they work well enough to sketch in. You can see in the image above I’ve finished one, and the larger, square one is almost completely full as well.

Once my current handmade sketchbooks are full, my next goal will be to take an old and cheap hardcover book, rip out the innards, and fill it with my own sketching/painting paper. Sea Lemon has a video on case binding, which looks to be exactly what I want to do. First, though, I need to fill up the ones I’ve already made.

All in all, I was surprised how easy it was to make a basic sketchbook that I could start filling with my own sketches and paintings. I’ve already made a half a dozen sketchbooks, and I still have plenty of paper and supplies for another twenty, all for the cost of a couple of professional sketchbooks.

A few more resources:

2017-12-12 — #art #bookbinding #diy #sketching