The fileinput module is a part of the standard library and is used when someone needs to iterate the contents of multiple files simultaneously. Well, Python's in-built open() function can also be used for iterating the content but for only one file at a time.
You'll explore the classes and functions provided by the fileinput module to iterate over multiple files.
But one thing, you could use fileinput to iterate the single file also, but it would be better to use the open() function for it.
Basic Usage
import fileinput
stream = fileinput.input(files=('test.txt',
'sample.txt',
'index.html'))
for data in stream:
print(data)
The fileinput module was first imported, and then the fileinput instance was created by calling fileinput.input() and passing the tuple of files (test.txt, sample.txt, and index.html). This will result in the return of an iterator.
The contents of the files were then iterated and printed using the for loop.
Hi, I am a test file.
Hi, I am a sample file for testing.
<html lang="en">
<head>
<title>Test HTML File</title>
</head>
<body>
<h1>Hi, I am a simple HTML File.</h1>
</body>
</html>
Another approach would be to use the fileinput module as a context manager. This method is somewhat safe because it ensures that the fileinput instance is closed even if an exception occurs.
import fileinput
with fileinput.input(files=('test.txt', 'sample.txt')) as files:
for data in files:
print(data)
In the above demonstration, the fileinput module was used as a context manager with the 'with' statement.
The above code will return an iterator and will assign it to the files variable (due to the as clause) then the data will be iterated using the files variable.
Hi, I am a test file.
Hi, I am a sample file for testing.
The fileinput.input() function is the primary interface of the fileinput module, by using it, the purpose of using the fileinput module is nearly fulfilled. You saw a glimpse of the fileinput.input() function in the previous section, this time, you'll learn more about it.
Syntax
fileinput.input(files=None, inplace=False, backup='', mode='r', openhook=None, encoding=None, errors=None)
Parameters:
files: Defaults to None. Takes a single file or multiple files to be processed.
inplace: Defaults to False. When set to True, the files can be modified directly.
backup: Defaults to an empty string. The extension is specified for the backup files when inplace is set to True.
mode: Default to read mode. This can only open files in read mode hence, we can open the file in r, rb, rU, and U.
openhook: Defaults to None. A custom function for controlling how files are opened.
encoding: Defaults to None. Specifies the encoding to be used to read the files.
errors: Defaults to None. Specifies how the errors should be handled.
Modifying the Files Before Reading
import fileinput
with fileinput.input(files=('test.txt', 'sample.txt'), inplace=True) as files:
for data in files:
modified_content = data.lower()
print(modified_content)
The parameter inplace is set to True in the above code, which enables the editing of the file before reading.
The upper code will lowercase the content present inside both files (test.txt and sample.txt).
Storing Backup of Files
When the inplace parameter is set to True, the original files can be edited, but the original state of the files can be saved in another file using the backup parameter.
import fileinput
with fileinput.input(files=('test.txt', 'sample.txt'),
inplace=True, backup='.bak') as files:
for data in files:
modified_content = data.capitalize()
print(modified_content)
The above code will capitalize the content and the original files will be saved as test.txt.bak and sample.txt.bak due to the backup='.bak'.
Controlling the Opening of the File
import fileinput
def custom_open(filename, mode):
data = open(filename, "a+")
data.write(" Data added through function.")
return open(filename, mode)
with fileinput.input(files=("test.txt", "sample.txt"), openhook=custom_open) as file:
for data in file:
print(data)
The custom_open() function is defined that takes two parameters filename and mode. The function opens the file in append + read mode and then writes the string and returns the file object.
The hook must be a function that takes two arguments, filename and mode, and returns an accordingly opened file-like object.Source
The files are then passed to the fileinput.input() function, and the openhook parameter is set to custom_open. The custom_open() function will be in charge of opening the files. The file content was iterated and printed.
Hi, i am a test file. Data added through function.
Hi, i am a sample file for testing. Data added through function.
Reading Unicode Characters
You have a file having Unicode characters and need to read that file, to read Unicode characters, specific encodings are used.
with fileinput.input(files=('test_unicode.txt'), encoding='utf-8') as files:
for data in files:
print(data)
The UTF-8 encoding can be used to read the Unicode characters, hence, the encoding parameter is set to utf-8 encoding.
😁😂😅
Handling Errors
To handle the error, use the errors parameter. Take the above code as an example: if the encoding was not specified, the code would throw a UnicodeError.
with fileinput.input(files=('test_bin.txt'), errors='ignore') as files:
for data in files:
print(data)
----------
ðŸ˜ðŸ˜‚😅
The errors parameter is set to ignore, which means that the error will be ignored. The errors parameter can also be set to strict (raise an exception if an error occurs) or replace (replace an error with a specified error).
There are some functions that can be used to access the information of the input files which are being processed using the fileinput.input() function.
Getting the File Names
Using the fileinput.filename() function, the name of the currently processed files can be displayed.
with fileinput.input(files=('test.txt', 'sample.txt')) as files:
for data in files:
print(f"File: {fileinput.filename()}")
print(data)
Output
File: test.txt
Hi, i am a test file. Data added through function. Added data to the file.
File: sample.txt
Hi, i am a sample file for testing. Data added through function. Added data to the file.
Getting the File Descriptor and Line and File Line Number
The fileinput.fileno() function returns the active file's file descriptor, the fileinput.lineno() function returns the cumulative line number, and the fileinput.filelineno() function returns the line number of the currently processed file.
with fileinput.input(files=('test.txt', 'sample.txt')) as files:
for data in files:
print(f"{fileinput.filename()}'s File Descriptor: {fileinput.fileno()}")
print(f"{fileinput.filename()}'s File Line Number: {fileinput.filelineno()}")
print(f"{fileinput.filename()}'s File Cumulative Line No.: {fileinput.lineno()}")
Output
test.txt's File Descriptor: 3
test.txt's File Line Number: 1
test.txt's File Cumulative Line No.: 1
sample.txt's File Descriptor: 3
sample.txt's File Line Number: 1
sample.txt's File Cumulative Line No.: 2
Checking Reading Status
with fileinput.input(files=('test.txt', 'sample.txt')) as files:
for data in files:
print(f"Read First Line: {fileinput.isfirstline()}")
print(f"Last Line Read From sys.stdin: {fileinput.isstdin()}")
----------
Read First Line: True
Last Line Read From sys.stdin: False
Read First Line: True
Last Line Read From sys.stdin: False
The fileinput.isfirstline() function returns True if the line read from the current file is the first line otherwise returns False, since both files contain a single line, it returned True.
When the last line of the input file was read from sys.stdin, the fileinput.isstdin() function returns True, otherwise, it returns False.
Closing the File
When using fileinput.input() function as the context manager with the with statement, the file closes anyway but fileinput.close() function is also used to close the resources when the work is done.
import fileinput
with fileinput.input(files=('test.txt', 'sample.txt')) as file:
for data in file:
if data > data[:26]:
fileinput.close()
print('File has more than 25 characters.')
else:
print(data)
The above code demonstrates the use of the fileinput.close() function, which closes the file if it contains more than 25 characters and prints a message otherwise the content is printed.
File has more than 25 characters.
However, because the file contained more than 25 characters, the file was closed and the message was printed.
The fileinput.FileInput class is an object-oriented alternative to the fileinput.input() function. The parameters are identical to those of the input() function.
Syntax
fileinput.FileInput(files=None, inplace=False, backup='', mode='r', openhook=None, encoding=None, errors=None)
Example
import fileinput
class OpenMultipleFiles:
def __init__(self, *args):
self.args = args
def custom_open(self, filename, mode):
data = open(filename, "a+")
data.write(" Added data to the file.")
return open(filename, mode)
def read(self):
with fileinput.FileInput(files=(self.args), openhook=OpenMultipleFiles().custom_open) as file:
for data in file:
print(data)
obj = OpenMultipleFiles('test.txt', 'sample.txt')
obj.read()
The class OpenMultipleFiles is defined in the above code. The class has an __init__ method that takes variadic arguments.
A custom_open method is defined within the class that opens the file in append+read mode, writes some data to the file, and returns the file object.
The read method is defined and within the read method the instance of the fileinput.FileInput is created and passed the self.args as the files argument and the openhook parameter is set to OpenMultipleFiles().custom_open. The contents of the files are then iterated and printed.
Finally, the OpenMultipleFiles class instance is created and passed the file names (test.txt and sample.txt) and stored within the obj variable. The read method is then invoked on the obj to read the specified files.
Hi, i am a test file. Data added through function. Added data to the file.
Hi, i am a sample file for testing. Data added through function. Added data to the file.
Comparison
Let's see how long it takes to process the contents of multiple files at the same time using the open() and the fileinput.input() function.
import timeit
code = '''
with open('test.txt') as f1, open('sample.txt') as f2:
f1.read()
f2.read()
'''
print(f"Open Function Benchmark: {timeit.timeit(stmt=code, number=1000)}")
setup = 'import fileinput'
code = '''
with fileinput.input(files=('test.txt', 'sample.txt')) as file:
for data in file:
data
'''
print(f"Fileinput Benchmark: {timeit.timeit(setup=setup, stmt=code, number=1000)}")
Using the timeit module, the above code measures the time it takes to process the contents of multiple files 1000 times for the fileinput.input() function and open() function. This method will aid in determining which is more efficient.
Open Function Benchmark: 0.3948998999840114
Fileinput Benchmark: 0.4962893000047188
Limitations
Every module is powerful in its own right, but it also has limitations, such as the fileinput module.
It does not read files, instead, it iterates through the contents of the file line by line and prints the results.
Cannot write or append the data into the files.
Cannot perform advanced file-handling operations.
Less performant because the program's performance may suffer when processing large files.
Conclusion
The fileinput module provides functions to process one or more than one file line by line to read the content. The fileinput.input() function is the primary interface of the fileinput module, and it provides parameters to give you more control over how the files are processed.
Let's recall what you've learned:
An overview of the fileinput module
Basic usage of the fileinput.input() with and without context manager
The fileinput.input() function and its parameters with examples
A glimpse of FileInput class
Comparison of fileinput.input() function with open() function for processing multiple files simultaneously
Some limitations of the fileinput module
🏆Other articles you might be interested in if you liked this one
✅How to use assert statements for debugging in Python?
✅Difference between the __init__ and __new__ methods.
✅What is context manager and the 'with' statement in Python?
✅How to implement getitem, setitem, and delitem in Python classes?
✅How to perform unit testing using the unittest module in Python?
✅File handling in Python - Opening, Reading, and much more.
✅Public, Protected, and Private access modifiers in Python.
That's all for now
Keep coding✌✌
