Working with the filesystem is an essential part of many programming tasks, especially when dealing with file input and output operations. Python provides several modules that can be used to interact with the filesystem, such as os, pathlib, and shutil. In this article, we will focus on the OS module, shutil module, and some basic file operations which provides a higher-level interface for working with files and directories.

Before diving into the specifics of the modules, let’s first discuss the basics of the filesystem in Python. A filesystem is a hierarchy of directories and files that is organized in a specific way. The top-level directory is called the root directory, which is represented by a forward slash ("/") on Unix-based systems or a backslash ("") on Windows-based systems.

Working with Files

Files are a crucial component of any computer program, and Python provides a set of functions and methods to handle them efficiently. Files are used to store and retrieve data, and Python can work with different types of files, including text files, binary files, CSV files, and more. In this article, we will explore the basics of working with files in Python, including opening, reading, writing, and closing files.

Opening a File

To interact with the filesystem in Python, we use the open() function to open files, and the os and pathlib modules to navigate directories and files. The os module provides a low-level interface to the filesystem, whereas the pathlib module provides a high-level interface that is easier to use.

Before we can work with a file in Python, we need to open it. To open a file, we use the open() function, which takes two arguments: the file name and the mode. The file name is a string that specifies the name and location of the file, while the mode specifies how we want to interact with the file.

In Python, files can be opened and manipulated in various modes using the built-in open() function. The mode parameter determines how the file is opened and what operations are allowed on it. Here are some of the common file modes in Python:

1. ‘r’ (Read):

   • This mode is used for reading files. It opens the file for reading only.
   • If the file does not exist, it raises a FileNotFoundError.
   • Example: file = open('filename.txt', 'r')

2. ‘w’ (Write):

   • This mode is used for writing to files. It removes the content of the file if it already exists and opens it for writing.
   • If the file does not exist, it creates a new empty file.
   • Example: file = open('filename.txt', 'w')

3. ‘a’ (Append):

   • This mode is used for appending data to an existing file.
   • It does not truncate the file; it opens it for writing at the end, so new data is added after the existing content.
   • If the file does not exist, it creates a new empty file.
   • Example: file = open('filename.txt', 'a')

4. ‘x’ (Exclusive creation):

   • This mode is used to create a new file but raises an error if the file already exists.
   • Example: file = open('filename.txt', 'x')

5. ‘b’ (Binary mode):

   • This mode is used in conjunction with the other modes (‘r’, ‘w’, ‘a’, ‘x’) to indicate that the file should be treated as a binary file.
   • Example: file = open('filename.txt', 'rb')

6. ’t’ (Text mode, default):

   • This mode is used in conjunction with the other modes to indicate that the file should be treated as a text file (default behavior).
   • Example: file = open('filename.txt', 'wt')

7. ’+’ (Read and Write):

   • This mode is used for both reading and writing. It allows you to perform both read and write operations on the file.
   • Example: file = open('filename.txt', 'r+')

8. ‘U’ (Universal newline):

   • This mode is used to open files in Universal Newline mode, which enables uniform newline handling (’\n’ or ‘\r\n’) when reading text files.

Here’s an example of opening a file in read mode:

1

file = open('example.txt', 'r')

It’s important to remember to close the file after you’re done with it using the close() method to free up system resources. Alternatively, you can use the with statement (context manager) to automatically close the file when you’re done, ensuring that resources are properly managed. Here’s an example of using the with statement:

1
2
3

with open('filename.txt', 'r') as file:
    # Perform file operations here
# File is automatically closed when exiting the 'with' block

Using the with statement is a recommended and safer way to work with files in Python. This is explained below in detail.

Reading a File

Once we have opened a file, we can read its contents using the read() method. The read() method reads the entire contents of the file and returns them as a string.

1
2
3

file = open('example.txt', 'r')
contents = file.read()
print(contents)

We can also read the file line by line using the readline() method. The readline() method reads one line of the file at a time and returns it as a string.

1
2
3
4
5

file = open('example.txt', 'r')
line = file.readline()
while line:
    print(line)
    line = file.readline()

Writing to a File

To write to a file in Python, we need to open the file in write or append mode. In write mode, the write() method overwrites the contents of the file, while in append mode, the write() method appends the new data to the end of the file.

Here’s an example of writing to a file in write mode:

1
2
3

file = open('example.txt', 'w')
file.write('Hello, World!')
file.close()

And here’s an example of writing to a file in append mode:

1
2
3

file = open('example.txt', 'a')
file.write('Hello again, World!')
file.close()

Closing a File

It’s important to close a file after we have finished working with it. To close a file, we use the close() method.

1
2
3

file = open('example.txt', 'r')
contents = file.read()
file.close()

Working with Files using the with Statement

Another way to work with files in Python is to use the with statement. The with statement automatically takes care of closing the file when we’re done with it.

1
2

with open('example.txt', 'r') as file:
    contents = file.read()

Working with files is an essential part of programming, and Python provides us with a simple and efficient way to do it. We can open, read, write, and close files using the built-in functions and methods. We can also work with files using the with statement, which takes care of closing the file automatically. By mastering the basics of working with files in Python

Working with OS module

The os module in Python provides a set of functions to work with the operating system, including file system operations. One of the key features of the os module is the ability to work with file paths. In this article, we will explore the basics of working with file paths using the os.path module, including joining and splitting paths, checking if a file exists, and more.

Joining Paths

The os.path.join() function is used to join one or more path components intelligently. This function is used to build a path string from separate strings representing the different directories and file names.

1
2
3
4

import os

path = os.path.join('C:', 'Users', 'User', 'Documents', 'example.txt')
print(path)

This code will output the following path:

1

C:\Users\User\Documents\example.txt

Splitting Paths

The os.path.split() function is used to split a path into its directory and file components. This function returns a tuple containing the directory path and the file name.

1
2
3
4
5
6
7

import os

path = 'C:/Users/User/Documents/example.txt'
# Here the os.path.split(path) will return two values. C:/Users/User/Documents and example.txt. dir_path contains C:/Users/User/Documents and file_name contains example.txt values. You can use any variables names in the place of dir_path and file_name.
dir_path, file_name = os.path.split(path)
print('Directory:', dir_path)
print('File:', file_name)

This code will output the following:

1
2

Directory: C:/Users/User/Documents
File: example.txt

Checking if a File Exists

The os.path.exists() function is used to check if a file or directory exists. This function returns True if the file or directory exists, and False otherwise.

1
2
3
4
5
6
7

import os

file_path = 'example.txt'
if os.path.exists(file_path):
    print('File exists')
else:
    print('File does not exist')

This code will output either “File exists” or “File does not exist”, depending on whether the file exists or not.

Getting the File Extension

The os.path.splitext() function is used to split a path into its base and extension components. This function returns a tuple containing the base name and the extension.

1
2
3
4
5
6

import os

file_path = 'example.txt'
file_name, file_ext = os.path.splitext(file_path)
print('File name:', file_name)
print('File extension:', file_ext)

This code will output the following:

1
2

File name: example
File extension: .txt

Getting the Absolute Path

The os.path.abspath() function is used to return the absolute path of a file or directory. This function takes a relative path as an argument and returns its absolute path.

1
2
3
4
5

import os

file_path = 'example.txt'
abs_path = os.path.abspath(file_path)
print('Absolute path:', abs_path)

This code will output the absolute path of the example.txt file.

The os.path module in Python provides a set of functions to work with file paths. By using these functions, we can join and split paths, check if a file or directory exists, get the file extension, and get the absolute path. These functions are essential for working with files and directories in a platform-independent way.

Working with filesytem shell mothods using shutil

The shutil module is a part of the standard library in Python, which means that it is available by default and does not require installation. It provides functions for copying, moving, and deleting files and directories, as well as archiving and compressing them.

Now let’s explore the shutil module and its methods.

1. Copying files and directories The shutil module provides two functions for copying files and directories: copy() and copy2(). The copy() function copies the file’s contents to a new file, while the copy2() function copies the file’s contents and metadata to a new file.

Example:

1
2
3
4
5
6
7

import shutil

# Copying a file
shutil.copy('example.txt', 'newfile.txt')

# Copying a directory
shutil.copytree('example_dir', 'new_dir')

2. Moving files and directories The shutil module provides two functions for moving files and directories: move() and rmtree(). The move() function moves the file or directory to a new location, while the rmtree() function removes the directory and all its contents.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import shutil

# Moving a file
shutil.move('example.txt', 'new_location/example.txt')

# Moving a directory
shutil.move('example_dir', 'new_location/example_dir')

# Removing a directory and all its contents
shutil.rmtree('example_dir')

3. Archiving and compressing files and directories The shutil module provides functions for creating and extracting archive files, as well as compressing and decompressing files. The supported archive formats are zip, tar, and gztar.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import shutil

# Creating a zip archive
shutil.make_archive('archive', 'zip', 'example_dir')

# Extracting a zip archive
shutil.unpack_archive('archive.zip', 'extracted_dir')

# Compressing a file
shutil.compress('example.txt', 'example.txt.gz')

# Decompressing a file
shutil.decompress('example.txt.gz', 'example.txt')

4. Other methods The shutil module also provides several other methods for working with files and directories, such as chown(), which changes the owner and group of a file or directory, and disk_usage(), which returns the total, used, and free disk space of a filesystem.

Example:

1
2
3
4
5
6
7

import shutil

# Changing the owner and group of a file
shutil.chown('example.txt', 'username', 'groupname')

# Getting the total, used, and free disk space of a filesystem
total, used, free = shutil.disk_usage('/')

The shutil module provides a high-level interface for working with files and directories in Python. It simplifies common operations such as copying, moving, and deleting files and directories, as well as archiving and compressing them. By using the shutil module, we can avoid the low-level details of interacting with the filesystem and focus on our specific task.

The shutil module is a powerful tool for working with the filesystem in Python. It provides several useful methods that simplify common filesystem operations and can save us a lot of time and effort. By understanding the basics of the filesystem in Python and using the shutil module effectively, we can efficiently manage our files and directories and achieve our desired results.

Suggested Article

If you’d like to continue reading, checkout our other tech articles here or browse all our topics here.

Conclusion

In conclusion, working with files, OS path utilities, and shell methods are all important aspects of programming in Python. The ability to read from and write to files, manipulate paths and directories, and execute shell commands and external programs can greatly enhance the functionality and flexibility of your Python scripts. By using the built-in modules and functions provided by Python, you can easily handle these tasks in a way that is efficient and reliable. With the knowledge gained from these articles, you should be well-equipped to work with files, OS path utilities, and shell methods in your own Python projects.