with

That may sound like an odd title for a tutorial, but it will make sense soon enough. 😉

Today, we will be learning about the with statement (or handle as I call it), how you use it, and why you should always use it no matter what.

If you already know about the concepts and functions in this tutorial, you may be thinking to yourself:

“Everybody already knows about this stuff. I see no need for you to write a tutorial on it.”

Ah, but that is not true. In the last few months before I began writing this tutorial (August 2013), I personally have seen brand new code not use the with statement. Fortunately, I had connections to the developers and could advise them on this matter, but that goes to show that this mighty feature is not taught as much as it should. That is why I am writing this tutorial, so the power of with may be known more widespread.

Moreover, there are some Python tutorials that do not teach about with, such as the Python tutorial at After Hours Programming and Learn Python The Hard Way. Not teaching about with does not discredit the other content, but helps reenforce the concept of cross-referencing.

Fortunately my favorite Python book, A Byte of Python, teaches to use with so that is a plus from my standpoint. 🙂

So, what is with, and why should you use it? Furthermore, what in the world is with? It is hard to define (at least for me, others may be able to define it better), but I will make an attempt anyway:

“The with statement is the easiest and best way to read and write files, as well as compressed archives such as Zip, Tar, and Gzip.”

That did not go well. 😛

Truly, in this tutorial, it is superior for you and me if we jump straight into example usage, and explain the advantages it had along the way.

While all code should be Python 2 and 3 compatible unless otherwise noted, any Exceptions are from Python 3.3+.

The usual way to open a file is to define a variable to the open() function which opens your file in the desired mode, be it reading, writing, or appending, read/write the file (either entire thing or certain line(s)), close it using close() and move on. Some example code for what I described would be:

This is all fine and dandy, until you hit one of two things: you forget to close it, or you experience an Exception. Then you’re in trouble.

Lets go with the former example, and transition into the latter along the way

By forgetting to close your file, the file is always open, and is always being used by the system. If you later on try to re-read/write that file or copy/move/delete it, you may be unable to since it is still open. If you have you perform a KeyboardInterrupt during execution because of an error in your code, the file will still be open, and will remain so until you restart your computer.

These same consequences can occur if you run into an Exception, the most common ones being a PermissionError, a FileNotFoundError, or a FileAlreadyExistsError. The last two can usually be avoided by using os.path.exists, but PermissionError has to be caught using a try...except...finally block. If we tried to write a file using the example above in a location we do not have permission to, such as Program Files, then the file will not be closed, and we will experience the same consequences as if we forgot to close it. Thus, our patched code using the aforementioned try...except...finally block looks like this:

Suddenly our small 3 lines of code has turned into 10 lines of code, and this catches only one Exception; catching more will require more except blocks. While there is nothing wrong with try...except...finally blocks, if you are extensively reading/writing files in your program, your code will get very large very quick. There really must be an easier and shorter way to do this, right?

Yes, there is! Enter with. 😀

Enter! Return! Get in here!

The with statement is like an automatic try...except...finally block, but written on the C code level (the language the standard Python implementation is written in). It’s essentially is a try...except...finally block, with the file close() function in the finally block, so it is always executed. It works the same way, but you don’t have to write it out yourself, you just use with! This is why I call it the with handle: it handles the file closure for you! (Read PEP 0343 if you want more technical information about with. It’s actually quite interesting.)

The syntax using this method is a bit different, but not majorly so.

As you can see, any file operations (read(), readlines(), write()) are indented one level, like how you preform operations using an if clause. Any non-file operations are preformed on same the indentation level as with. This is important! Many times have I caught myself performing non-file operations on the wrong indentation level, creating issues I couldn’t fix until I discovered the error. Sometimes, I’ve found I’ve written out a good chunk of code before catching my error!

Did you notice this is the same code from the first example? We’ve reduced it down to two lines, which is a tad shorter than the original, and (by extension), much better than our 10-liner script.

However, this does not provide an excuse to never use try...except...finally blocks!

The with handle only provides automatic file closing. If you have run into a PermissionError, you still have to catch and deal with it. The only difference is if you don’t have to run close() on your file.

Side Notes

Two more side notes, and this tutorial will be over.

First, if you want to check if your file is closed or not, you can run closed and check the result. It returns True if it is closed, and False if it is still open. This is handy if you insist on not using with (and I’ll scold you for doing that :P), or as a debugging tool.

The second thing is that Python 3 supports more usage of with than Python 2 does. Although with was first introduced during the Python 2.5 era so to speak (you could only access it using from __future__ import with_statement. It was made a default statement in 2.6, making the import obsolete), only lately has it caught on in general usage. In addition, archiving actions in modules such as tarfile, zipfile, and shutil were only updated for with in Python 3K (and not necessarily in 3.0). Thus, you will benefit from the with the most if you program using Python 3. If you stubbornly choose to code using Python 2, you only get file operations from with.

And there you have it! You now know about the with statement, what it is used for, how it works, and why you should always use it NO MATTER WHAT. From now on, if I catch you writing new code and not use with… let’s just say I had better not catch you not using with. 😛

Advertisements

Triangular Reactions

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s