# Perform High-Level File Operations In Python - shutil Module

Copying or moving files or folders manually from one directory to another directory could be a real pain. This can be automated using a Python module called `shutil`.

Shutil module provides some high-level operations on files and collection of files like copying, moving, or removing the files.

In other words, the `shutil` module helps in automating the task of file copying or moving from one directory to another directory. It comes under the Python standard library so it doesn't need to be installed externally.

In this article, we'll learn this module.

## Copy files to another directory

`shutil.copy()` method is used to copy the content of the ***source*** file to the **destination** file or directory. While the process of copying also copies the file's permission mode but other metadata like the file’s creation and modification times, is not preserved.

The source must represent a file but the destination can be a file or a directory. If the destination is a directory then the file will be copied to the destination directory with a base filename from the source. The destination must be writable.

If the destination is a file and already exists then it will be replaced by the source file otherwise the new file will be created.

**Syntax:** `shutil.copy(source, destination, *, follow_symlinks = True)`

**Parameter:**

* `source`: A string that shows the path of the source file.
    
* `destination`: A string that shows the path of the destination file or directory.
    
* `follow_symlinks` (optional): The default value of this parameter is `True`. If it is `False` and the source represents a symbolic link then the destination will be created as a symbolic link.
    

**Copying source file to the destination**

```python
import shutil

# source and destination path
src_path = "hello.txt"
dst_path = "hello2.txt"

path = shutil.copy(src_path, dst_path)

# Printing the path of destination dir
print('Destination Path:', path)
```

**Output**

```python
Destination Path: hello2.txt
```

**If the destination is the directory**

```python
import os
import shutil

# Making a directory
os.mkdir('Project')

# Listing the files in it
print("Empty Folder: ", os.listdir('Project'))

# source and destination path
src_path = "hello.txt"
dst_path = "Project"

# Copying file to destination
path = shutil.copy(src_path, dst_path)

# Printing the name of copied filename and path of destination dir
print('File Copied:', os.listdir('Project'))
print('Destination Path:', path)
```

**Output**

```python
Empty Folder:  []
File Copied: ['hello.txt']
Destination Path: Project\hello.txt
```

## Copy metadata along with the file

`shutil.copy2()` method is used to copy the contents of the source file to the destination file or directory. It is the same as the `shutil.copy()` method but the only difference is that **this method tries to preserve the metadata of the file**.

**Syntax:** `shutil.copy2(source, destination, *, follow_symlinks = True`)

**Parameter:**

* `source`: A string that shows the path of the source file.
    
* `destination`: A string that shows the path of the destination file or directory.
    
* `follow_symlinks` (optional): The default value of this parameter is `True`. If it is `False` and the source represents a symbolic link then it attempts to copy all metadata from the source symbolic link to the newly-created destination symbolic link. This functionality is platform dependent.
    

```python
import os
import shutil

# Specifying src and dst paths
src_path = "Data\geek.txt"
dst_path = "NewData"

print("Before copying the metadata:")

# Listing the files inside src folder
src_folder = "Data"
print(os.listdir(src_folder))

# Printing the status of the files inside src path
print("Metadata:", os.stat(src_path), "\n")

# Copying content from src to dst
dest = shutil.copy2(src_path, dst_path)

print("After copying the metadata:")

# Listing the files inside dst path
print(os.listdir(dst_path))

# Printing the status of the files inside dst path
print("Metadata:", os.stat(dest), "\n")

# Printing the destination path of newly created files
print("Files copied to:", dest)
```

**Output**

```python
Before copying the metadata:
['geek.txt', 'hello.py', 'python.txt']
Metadata: os.stat_result(st_mode=33206, st_ino=7036874417909405, st_dev=3836766283, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1662197106, st_mtime=1662196237, st_ctime=1662194099) 

After copying the metadata:
['geek.txt']
Metadata: os.stat_result(st_mode=33206, st_ino=20547673300020899, st_dev=3836766283, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1662197106, st_mtime=1662196237, st_ctime=1662204751) 

Files copied to: NewData\geek.txt
```

**If the destination is a directory**

```python
import os
import shutil

path = "NewData"

print("Before copying the files:")

# Listing the files inside dst folder
print(os.listdir(path), "\n")

# Specifying src and dst paths
src_path = "Sample/hello.txt"
dst_path = "NewData"

# Copying content from src to dst
dest = shutil.copy2(src_path, dst_path)

print("After copying the metadata:")

# Listing the files inside dst folder
print(os.listdir(path))

# Printing the destination path of newly created files
print("Files copied to:", dest)
```

**Output**

```python
Before copying the files:
[] 

After copying the files:
['hello.txt']
Files copied to: NewData\hello.txt
```

## Copy content from one file to another file

`shutil.copyfile()` method is used to copy the content of the source file to the destination file without the metadata. The **source** and **destination must represent a file** and the destination file must be writable. If the destination already exists then it'll be replaced or a new file will be created.

**Syntax:** `shutil.copyfile(source, destination, *, follow_symlinks = True)`

**Parameter:**

* `source`: A string that shows the path of the source file.
    
* `destination`: A string that shows the path of the destination file or directory.
    
* `follow_symlinks` (optional): The default value of this parameter is `True`. If `False` and source represent a symbolic link then a new symbolic link will be created instead of copying the file.
    

```python
import shutil

# Specifying the src and dest file paths
src_path = "Data/python.txt"
dst_path = "NewData/geek.txt"

# Copying the src file to dst file
dest = shutil.copyfile(src_path, dst_path)

# Printing the path of newly created file
print("File Copied to:", dest)
```

**Output**

```python
File Copied to: NewData/geek.txt
```

**If the source and destination file path is the same**

If it happens then a `SameFIleError` will be raised and we can handle the error by using `shutil.SameFileError`.

```python
import shutil

# Specifying the dst file same as src file
src_path = "Data/python.txt"
dst_path = "Data/python.txt"

try:
    dest = shutil.copyfile(src_path, dst_path)
    print("File Copied to:", dest)

# Printing error 
except shutil.SameFileError:
    print("Source and Destination represent the same file.")
```

**Output**

```python
Source and Destination represent the same file.
```

## Copy the directory tree

`shutil.copytree()` method is used to replicate the complete directory tree. It recursively copies the entire directory tree rooted at the ***source*** directory to the ***destination*** directory. The destination directory must not already exist. The new directory will be created during the process.

**Syntax:** `shutil.copytree(src, dst, symlinks = False, ignore = None, copy_function = copy2, ignore_dangling_symlinks = False)`

**Parameter:**

* `src`: A string that shows the path of the source directory.
    
* `dest`: A string that shows the path of the destination.
    
* `symlinks` (optional): This parameter accepts `True` or `False`, depending on which the metadata of the original links or linked links will be copied to the new tree.
    
* `ignore` (optional): If ignore is given, it must be a callable that will receive as its arguments the directory being visited by `copytree()`, and a list of its contents, as returned by `os.listdir()`.
    
* `copy_function` (optional): The default value of this parameter is `copy2`, other copy function like `copy()` can be used for this parameter.
    
* `ignore_dangling_symlinks` (optional): This parameter value when set to `True` is used to put a silence on the exception raised if the file pointed by the symlink doesn’t exist.
    

```python
import shutil

# Specifying src directory
src_path = "Data"
# src dir tree will be copied to the destination
dst_path = "Project"

# Copy the entire src directory tree to the destination
dest = shutil.copytree(src_path, dst_path)
print("File Copied to:", dest)
```

**Output**

```python
File Copied to: Project
```

## Remove a directory tree

`shutil.rmtree()` method is used to **delete the entire directory tree**, the specified path must point to a directory (but not to a symbolic link to a directory).

**Syntax:** `shutil.rmtree(path, ignore_errors=False, onerror=None)`

**Parameter:**

* `path`: A path-like object representing a file path. A path-like object is either a string or bytes object representing a path.
    
* `ignore_errors`: If `ignore_errors` is true, errors resulting from failed removals will be ignored.
    
* `oneerror`: If `ignore_errors` is false or omitted, such errors are handled by calling a handler specified by `onerror`.
    

**Removing a top-level directory tree**

```python
import shutil

# Directory tree to be removed
dir_to_be_removed = "Sample"

# Removing a top-level dir
remove = shutil.rmtree(dir_to_be_removed)

print("Successfully removed.")
```

**Output**

```python
Successfully removed.
```

**Removing a directory tree under a directory**

```python
import shutil
import os

# Directory location
path = "Sample/NewDirectory"

# directory to be removed
dir_to_be_removed = "Python"

# joined the path
join_path = os.path.join(path, dir_to_be_removed)

# removing directory
shutil.rmtree(join_path)
print("Successfully removed.")
```

**Output**

```python
Successfully removed.
```

## Move files to another location

`shutil.move()` method is used to move the source file or a directory to the destination location. If the destination directory already exists then the source directory will move inside that directory.

**Syntax:** `shutil.move(src, dst, copy_function=copy2)`

**Parameter:**

`src`: A string that shows the path of the source file.

`dst`: A string that shows the path of the destination file or directory.

`copy_function`(optional): The default value of this parameter is `copy2`. We can use other copy functions also.

```python
import shutil

# Specifying src and dst directories
src_path = "Project"
dst_path = "NewProject"

# Moving src dir to dst dir
dest = shutil.move(src_path, dst_path)

# Printing the destination location
print("Directory moved to:", dest)
```

**Output**

```python
Directory moved to: NewProject\Project
```

**If the destination doesn't exist**

```python
import shutil

# Specifying src directory
src_path = "New_dir"
# Non-existing directory
dst_path = "Moved_dir"

# Moving src dir to dst dir
dest = shutil.move(src_path, dst_path)

# Printing the destination location
print("Directory moved to:", dest)
```

**Output**

```python
Directory moved to: Moved_dir
```

## Finding executable files

`shutil.which()` method is used to find the path to an executable file specified to the `cmd` argument. If no `cmd` is specified then it will return `None`.

**Syntax:** `shutil.which(cmd, mode = os.F_OK | os.X_OK, path = None)`

**Parameter:**

* `cmd`: File in string format.
    
* `mode`: This parameter specifies the mode by which the method should execute.
    
* `os.F_OK` tests the existence of the path
    
* `os.X_OK` checks if the path can be executed
    
* `path`: This parameter specifies the path to be used, if no path is specified then the results of `os.environ()` are used.
    

```python
import shutil

# Executable file to be located
exec_file = "pip"

# Finding the location
location = shutil.which(exec_file)

# Printing the location of the executable file
print(location)
```

**Output**

```python
D:\SACHIN\Python310\Scripts\pip.EXE
```

## More cool methods

There are some other cool methods that the `shutil` module provides other than the above-mentioned methods.

### Copy the stats of a file or a directory

`shutil.copystat()` method is used to copy the stats of a file or a directory like permission bits, last access time, last modification time, and flags from the **source** to the **destination**.

```python
import shutil
import os
import time

src_path = "D:/SACHIN/Pycharm/Number-Converter"
dst_path = "D:/SACHIN/Pycharm/OpenCV/project1.py"

# Printing metadata of source dir
print("Source Data")
print("Modified at:", time.ctime(os.stat(src_path).st_mtime))
print("Accessed at:", time.ctime(os.stat(src_path).st_atime), "\n")

# Printing metadata of destination dir
print("Destination Data")
print("Modified at:", time.ctime(os.stat(dst_path).st_mtime))
print("Accessed at:", time.ctime(os.stat(dst_path).st_atime), "\n")

# Copying stats of src to dst
shutil.copystat(src_path, dst_path)

# Printing metadata after using shutil.copystat() method
print("Destination data after using shutil.stat()")
print("Modified at:", time.ctime(os.stat(dst_path).st_mtime))
print("Accessed at:", time.ctime(os.stat(dst_path).st_atime))
```

**Output**

```python
Source Data
Modified at: Fri Jun 10 19:31:51 2022
Accessed at: Mon Sep  5 16:22:19 2022 

Destination Data
Modified at: Tue Jul 27 19:00:02 2021
Accessed at: Mon Sep  5 16:23:27 2022 

Destination data after using shutil.stat()
Modified at: Fri Jun 10 19:31:51 2022
Accessed at: Mon Sep  5 16:22:19 2022
```

### Retrieve disk usage stats

`shutil.disk_usage(path)` method is used to get the **statistics of disk usage** of the specified **path**. The return value has attributes **total**, **used**, and **free** which are the amount(in bytes) of the total, used, and free space. A ***path*** may be a file or a directory.

```python
import shutil

# Checking the stats
disk_size = shutil.disk_usage("NewSample")

# Printing the stats
print("The stats are:", disk_size)
```

**Output**

```python
The stats are: usage(total=374277664768, used=33221120000, free=341056544768)
```

### Archive a directory

`shutil.make_archive()` is used to create an archive file in a `zip` or `tar` format.

```python
import shutil

# Path of directory to be archived
directory = "NewSample"

# Archiving the directory with zip format
archive = shutil.make_archive(directory, format="zip")

print(archive)
```

**Output**

```python
NewSample.zip
```

### Unpacking a directory

`shutil.unpack_archive()` is used to unpack the archive file. The method takes two arguments, the first is the `filename` which is the **full path of the archive file** and the second is the `directory` in which the files are extracted.

```python
import shutil

# Path of directory to be unpacked
directory = "NewSample.zip"

# Unpacking the archive dir
shutil.unpack_archive(directory, "extracted_dir")

print(f"Successfully extracted the files from {directory}.")
```

**Output**

```python
Successfully extracted the files from NewSample.zip.
```

> For performing the high-level archiving operations on files in Python, See [zipfile](https://geekpython.in/zipfile-read-and-write-zip-files-without-extracting-it-in-python) module.

## Conclusion

We sure learned to handle high-level file operations using the `shutil` module. We've learned how to

* Copy the file or a directory to another directory
    
* Copy the metadata of the file
    
* Permanently move the file or a directory to another location
    
* Remove and copy the directory tree
    

Along with these operations, we've seen some of the cool methods which are provided by the `shutil` module.

**Reference:** https://docs.python.org/3.10/library/shutil.html

---

**That's all for now**

**Keep Coding✌✌**