Read, write, and create files in Python (with and open())
In Python, the open() function allows you to read a file as a string or list, and create, overwrite, or append a file.
Read and write files with open() and with
For both reading and writing scenarios, use the built-in open() function to open the file.
The file object, indicated by the path string specified in the first argument, is opened. Use the mode argument to specify read or write, text or binary. Details are described below.
Paths can be either absolute or relative to the current directory. You can check the current directory with os.getcwd() and change it with os.chdir().
Text files are read as io.TextIOWrapper object.
path = 'data/src/test.txt'
f = open(path)
print(type(f))
# <class '_io.TextIOWrapper'>
f.close()
As shown in the example above, you need to close the file object using the close() method. Alternatively, a with block automatically closes the file when the block ends, providing a more convenient approach.
with open(path) as f:
print(type(f))
# <class '_io.TextIOWrapper'>
You can use any variable name for the xxx in with open() as xxx:. It represents a file object opened with open(), named xxx, and used within the block. While f is commonly used, other names are also acceptable.
Encoding specification: encoding
Specify the encoding for reading or writing text files with the encoding argument of open(). The encoding string can be in uppercase or lowercase and can use either a hyphen - or an underscore _. For example, both 'UTF-8' and 'utf_8' are allowed.
Refer to the official documentation for the encodings supported by Python.
The default value of encoding depends on the platform. You can check it with locale.getpreferredencoding().
import locale
print(locale.getpreferredencoding())
# UTF-8
Read text files
Open a file for reading: mode='r'
To open a file for reading, set the mode argument of open() to 'r'. The default value for the mode argument is 'r', so you can omit it. In the following examples, it is omitted.
If you specify a non-existent path in mode='r', you will get an error (FileNotFoundError).
# with open('data/src/test_error.txt') as f:
# print(type(f))
# FileNotFoundError: [Errno 2] No such file or directory: 'data/src/test_error.txt'
Read the entire file as a string: read()
To read the entire file into a single string, use the read() method on the file object.
with open(path) as f:
s = f.read()
print(type(s))
print(s)
# <class 'str'>
# line 1
# line 2
# line 3
Although the file object is closed at the end of the with block, the assigned variable remains accessible outside the block.
with open(path) as f:
s = f.read()
print(s)
# line 1
# line 2
# line 3
Read the entire file as a list: readlines()
To read the entire file as a list of lines, use the readlines() method. All lines except the last one include a newline character \n at the end.
with open(path) as f:
l = f.readlines()
print(type(l))
print(l)
# <class 'list'>
# ['line 1\n', 'line 2\n', 'line 3']
If you want to remove the trailing newline character, you can use a list comprehension and call rstrip() on each element.
with open(path) as f:
l_strip = [s.rstrip() for s in f.readlines()]
print(l_strip)
# ['line 1', 'line 2', 'line 3']
Read a file line by line: readline()
When iterating over the file object with a for loop, you can retrieve each line as a string (including the newline character at the end). Here, repr() is used to display the newline character as is.
with open(path) as f:
for s_line in f:
print(repr(s_line))
# 'line 1\n'
# 'line 2\n'
# 'line 3'
To read one line at a time, use next(). An error occurs if there are no more lines to read.
with open(path) as f:
print(repr(next(f)))
print(repr(next(f)))
print(repr(next(f)))
# print(repr(next(f)))
# StopIteration:
# 'line 1\n'
# 'line 2\n'
# 'line 3'
The readline() method of the file object can also retrieve one line at a time, but it does not raise an error after EOF (end of file) and continues to return an empty string ''.
with open(path) as f:
print(repr(f.readline()))
print(repr(f.readline()))
print(repr(f.readline()))
print(repr(f.readline()))
print(repr(f.readline()))
# 'line 1\n'
# 'line 2\n'
# 'line 3'
# ''
# ''
Write text files
Open a file for writing: mode='w'
To open a file for writing, set the mode argument of open() to 'w'. The file's contents will be overwritten if it exists, or a new file will be created if it does not.
Be careful not to specify a non-existent directory for the new file, as this will result in an error (FileNotFoundError).
# with open('data/src/new_dir/test_w.txt', mode='w') as f:
# f.write(s)
# FileNotFoundError: [Errno 2] No such file or directory: 'data/src/new_dir/test_w.txt'
Write a string: write()
To write a string to a file, use the write() method on the file object.
path_w = 'data/temp/test_w.txt'
s = 'New file'
with open(path_w, mode='w') as f:
f.write(s)
with open(path_w) as f:
print(f.read())
# New file
Strings containing newline characters are written without any modification.
s = 'New line 1\nNew line 2\nNew line 3'
with open(path_w, mode='w') as f:
f.write(s)
with open(path_w) as f:
print(f.read())
# New line 1
# New line 2
# New line 3
For more information on strings with line breaks, see the following article:
The write() method only accepts strings as arguments. Passing a different type, such as an integer (int) or a floating-point number (float), will result in a TypeError. To write a non-string value, convert it to a string using str() before passing it to write().
Write a list: writelines()
To write a list of strings to a file, use the writelines() method. Note that this method does not automatically insert line breaks.
l = ['One', 'Two', 'Three']
with open(path_w, mode='w') as f:
f.writelines(l)
with open(path_w) as f:
print(f.read())
# OneTwoThree
To write each element of the list on a separate line, create a string containing newline characters by using the join() method.
with open(path_w, mode='w') as f:
f.write('\n'.join(l))
with open(path_w) as f:
print(f.read())
# One
# Two
# Three
The writelines() method only accepts lists containing strings as elements. If you want to write a list with elements of other types, such as integers (int) or floating-point numbers (float), you need to convert the list to a list of strings. See the following article:
Create an empty file: pass
To create an empty file, open a new file in write mode (mode='w') without writing any content.
Since the with block requires some statement to be written, use the pass statement, which does nothing.
with open('temp/empty.txt', 'w'):
pass
Create a file only if it doesn't exist
Using mode='w' can accidentally overwrite an existing file.
To write to a file only if it doesn't exist, i.e., create a new file without overwriting, you can use one of the following two methods:
Open a file for exclusive creation: mode='x'
To create a new file only if it does not already exist, set the mode argument of open() to 'x'. If the specified file exists, a FileExistsError will be raised.
# with open(path_w, mode='x') as f:
# f.write(s)
# FileExistsError: [Errno 17] File exists: 'data/src/test_w.txt'
By using try and except for exception handling, you can create a new file if it doesn't exist and take no action if it already exists.
try:
with open(path_w, mode='x') as f:
f.write(s)
except FileExistsError:
pass
Check if the file exists before opening
Use the os.path.isfile() function from the os.path module in the standard library to check if a file exists.
import os
if not os.path.isfile(path_w):
with open(path_w, mode='w') as f:
f.write(s)
Append to a file
Open a file for appending: mode='a'
To open a file for appending, set the mode argument of open() to 'a'.
write() and writelines() will append to the end of an existing file.
with open(path_w, mode='a') as f:
f.write('Four')
with open(path_w) as f:
print(f.read())
# One
# Two
# ThreeFour
If you want to add a final line, include the newline character when appending.
with open(path_w, mode='a') as f:
f.write('\nFour')
with open(path_w) as f:
print(f.read())
# One
# Two
# ThreeFour
# Four
Note that if the file does not exist, it will be created as a new file, just like when mode='w' is used.
Insert at the beginning or in the middle
mode='r+'
When the mode argument of open() is set to 'r+', the file is opened in update mode.
write() and writelines() will overwrite the existing file from the beginning.
with open(path_w, mode='r+') as f:
f.write('12345')
with open(path_w) as f:
print(f.read())
# 12345wo
# ThreeFour
# Four
In the example above, One\nTwo is overwritten by 12345 from the beginning, resulting in 12345wo. The \n is treated as one character.
You can also move the position by using the seek() method of the file object, but you need to specify the position in characters, not lines. This will also overwrite.
with open(path_w, mode='r+') as f:
f.seek(3)
f.write('---')
with open(path_w) as f:
print(f.read())
# 123---o
# ThreeFour
# Four
readlines() and insert()
If the text file is not huge, it is easier to read the entire file as a list using readlines() and process it. The insert() method of the list allows you to insert new strings at the beginning or in the middle of the lines.
Read the file as a list with readlines(), insert an element using insert(), and then write the updated list using writelines().
You can specify the number of lines to insert in the first argument of insert().
with open(path_w) as f:
l = f.readlines()
l.insert(0, 'FIRST\n')
with open(path_w, mode='w') as f:
f.writelines(l)
with open(path_w) as f:
print(f.read())
# FIRST
# 123---o
# ThreeFour
# Four
For adding, inserting, and removing elements to a list, see the following articles:
- Add an item to a list in Python (append, extend, insert)
- Remove an item from a list in Python (clear, pop, remove, del)
Read and write binary files
Adding a b to the end of the mode argument enables reading and writing binary files. For example, mode='rb' is for reading binary files, and mode='ab' is for appending to the end of binary files.
Like text files, binary files also support read(), readline(), readlines(), write(), and writelines() as methods of the file object.
To open, process, and save image files, use libraries such as Pillow or OpenCV.