A Bug Report

I just fixed a Django bug that took me ages to solve. Part of why it took so long is that the errors I was getting weren’t surfacing the right search results. So, in case anyone ever has a similar problem, I will record what happened here and hope the search algorithms send this their way.

My project has a Vue-based front end that queries the Django backend fairly regularly for data. My tests started to fail non-deterministically, with DoesNotExist errors popping up here and there.

At the same time, but not always on the same tests, I started having difficulty cleanly finishing my tests. I’d get notifications that the database couldn’t be destroyed because something was still connected (ie “Got an error the test database: database is being accessed by other users; DETAIL:  There is 1 other session using the database.”)

I spent a long time trying to hunt down the DoesNotExist exceptions and eventually satisfied myself that it was some kind of database connection issue because Django would go from saying “here’s your object!” to “nope, doesn’t exist” literally on the next line. This fit with the problems destroying the database.

However, I don’t know very much about how Django manages databases. I started reading the docs in an effort to better understand what might be going wrong. I was briefly excited by this note that using LiveServerTestCase (which I was) caused tests to randomly fail (which they were) but then I saw it was only for Sqlite back ends, and I’m using postgres.

Eventually, many pages into a google search for answers, I found this blog post, which warns that every time a Django view queries the database, that query is wrapped in a transaction. If two queries happen at once, the second may fail since the row being queried is unreadable. The blog post sort of brushes off the solution of using a non_atomic_requests decorator, but since the problematic view is entirely read_only, that solution worked for me.