7 Easy Ways to Concatenate Files in Linux

Written by

Reviewed by

Last updated: July 24, 2024

Expert verified

SVG Image

TL;DR

To concatenate files in Linux, you can use these three popular command-line tools:

  1. cat command to combine or view files in a vertical sequence by running cat file1 file2 > newfile in the Linux command prompt.
  2. paste command to merge the contents of multiple files horizontally with paste file1 file2.
  3. join command to combine two files based on a common field, allowing for the creation of a merged output file with matching data from both input files using join -t, -1 1 -2 1 file1 file2 > output.

Check out the comprehensive guide below to learn more about how to concatenate files in Linux.

Combining multiple files into one can be a common need, especially when managing data, logs, or backups. If you’ve wondered how to join files efficiently in Linux, you’re in the right place. I’ll guide you through different methods to concatenate files in this post. I’ll also share best practices to ensure smooth and error-free operations. You’ll learn about commands like cat, tac, awk, sed, and find, and how to use each one for specific needs. Let’s explore and make your file management tasks in Linux easier!

What is File Concatenation?

File concatenation means joining the contents of multiple files into a single file. Imagine you have several text files, and you want to combine them into one long document. This process is called concatenation.

Why is File Concatenation Important in Linux?

  • Scripting and Automation: In Linux, you can automate file concatenation using scripts. This helps in performing repetitive tasks efficiently, saving time and reducing errors.
  • Easier Data Management: By combining multiple files into one, you simplify data handling. Instead of dealing with many smaller files, you have one comprehensive file.
  • Efficient Processing: Many tasks, like analyzing logs or processing data, become easier and faster when all information is in one file.
  • Backup and Archiving: Concatenating files can help in creating backups or archives. You can bundle multiple files into one for easier storage or transfer.

How to Concatenate Files in Linux

To concatenate files in Linux, you can use several commands like cat, tac, awk, sed, and find. The cat command is the most common and combines files by listing them and redirecting the output to a new file. For example, cat file1 file2 > newfile merges file1 and file2 into newfile.

In the following section, I will explore each of these methods in more detail to help you better understand how to use them:

1. cat Command

The cat command is the most commonly used command for vertical concatenation. That is, it adds the contents of the first file to the end of the second file. The output file contains all the content of both files, one after the other. Here’s how to use it:

  1. Create two files using gedit or any other text editor in Linux. Then, copy-paste the content below:

file1:

This text is from file 1.

file2:

This text is from file 2.
  1. Now, lunch the Terminal app and run the following command to concatenate these two files:
cat file1 file2 > newfile
  1. This command concatenates file1 and file2 into a new file called newfile. The > symbol redirects the output of the cat command to a new file.
Concatenate Files in Linux vertically

2. paste Command

The paste command is used for horizontal concatenation. This type of concatenation adds the contents of one file to the side of the other file. The output file would contain the content of both files in a single row. Here is an example of two files that you could use to understand this command:

  1. Create these files in a text editor to concatenate them using the paste command.

file1:

apple banana orange

file2:

red yellow orange
  1. After that, execute the command below in the Terminal app to concatenate these two files:
paste file1 file2
  1. When you use the paste command with these two files, it will combine the lines from both files and separate them by a delimiter, which is a tab by default. Here’s the output you would get:
horizontal concatenation of content

3. join Command

The join command is used to concatenate files in Linux that have a common field. I’ll create sample text files (file1 and file2) to demonstrate how this command works. Here are the steps to follow:

  1. Create file1 and file2 with the following text:

File1:

Employee Name, Department
John Doe, Sales
Jane Smith, Marketing
Bob Johnson, HR

File2:

Employee Name, Salary
John Doe, 70000
Jane Smith, 90000
Bob Johnson, 60000
  1. To join these two files based on the employee name field, you can use the following command in the Terminal app:
join -t, -1 1 -2 1 file1 file2 > output
  1. This command uses the -t option to specify that the delimiter between fields is a comma and the -1 and -2 options to specify that the join should be based on the first field in both files (which is the employee name field in this case).
  2. The resulting file, output.txt, will contain the concatenated data from both files based on the common field of Gender. It will include columns for Name, Age, Occupation, and Salary, with rows for each individual that appears in both files with the same gender.
concatenated data based on common field

4. tac Command

The tac command in Linux is used for concatenating and reversing the order of lines in a file. This command is particularly useful when you need to process files in reverse order, such as viewing the latest entries in a log file.

  1. Run the following command to concatenate files using tac command:
tac file1 file2 > newfile

This command will concatenate file1 and file2, reversing the order of lines, and save the output to newfile.

concatenating files using tac command

5. awk Command

The awk command is a powerful text-processing tool in Linux that allows for advanced concatenation of files with great flexibility and customization.

  1. In your terminal execute the following command to concatenate files:
awk 'FNR==1{print ""}{print}' file1 file2 > newfile

This command will concatenate file1 and file2 and add a newline between the contents of the files.

Explanation of the Command

  • FNR==1{print “”}: This part of the command ensures that a newline is printed before the content of each file except the first one.
  • {print}: This part prints each line of the files.
  • file1 file2 > newfile: This specifies the input files and the output file.
concatenating files using awk command

6. sed Command

The sed command, or stream editor, is a versatile tool for text manipulation in Linux. It can also be used for concatenating files.

  1. Type the following command for concatenating files using sed is:
sed '1r file2' file1 > newfile

This command concatenates file1 and file2, placing the contents of file2 before file1, and saves the output to newfile.

Syntax Breakdown

  • -e r file2: This option tells sed to read and insert the contents of file2.
  • file1: The main input file.
  • > newfile: The output file where the concatenated content will be saved.
concatenating files using sed command

7. find Command

The find command in Linux is used to search for files in a directory hierarchy. It can be combined with other commands to concatenate files located in subdirectories.

  1. Open your Terminal and run the following command:
find . -type f -exec cat {} + > newfile

This command finds all files in the current directory and its subdirectories, concatenates them, and saves the output to newfile.

Explanation of the Command

  • > newfile: Specifies the output file.
  • .: Specifies the current directory as the starting point for the search.
  • -type f: Restricts the search to files (excluding directories).
  • -exec cat {} +: Executes the cat command on the found files, with {} being a placeholder for the filenames.
concatenating files using find command

If you want to learn how to use find command in linux, read this detailed guide.

5 Best Practices to Concatenate Files in Linux

Now that I’ve covered the most popular command-line tools for concatenating files in Linux, let’s discuss the five best practices to follow when using these tools. Here’s the breakdown:

  • 📄 Use the correct file format: Ensure that all files being concatenated have the same format. Otherwise, you may experience issues with file encoding and formatting.
  • 💾 Backup your files: Before concatenating files, creating a backup of your original files is always a good idea. This way, you can restore your files if any issues arise during the concatenation process. Use the cp command to create a backup copy of your files.
  • 🔑 Check file permissions: Ensure you have the necessary permissions to access and modify the concatenated files. Otherwise, you may encounter errors during the concatenation process. Use the chmod command to adjust file permissions if needed.
  • Validate input: Before concatenating files on Linux, ensure the input is valid and error-free. This will help to prevent any issues with the concatenated file. You can use tools like grep, sed, or awk to filter and validate input data.
  • 🧹 Clean up files after concatenation: Once you’ve concatenated your files, clean up any temporary files or backups created during the process. This will help to keep your file system organized and avoid clutter. Use the rm command to remove unnecessary files after the concatenation process is complete.

Final Thoughts

In this article, I’ve explored various methods to concatenate files in Linux, such as using the cat, tac, awk, sed, and find commands. These methods offer flexibility and efficiency for different use cases.

If you found this guide helpful, I recommend:

Frequently Asked Questions

What are the limitations of file concatenation?

The main limitation of file concatenation is that it only involves combining two or more files into a single file. It has compatibility issues, file size limitations, loss of metadata, the potential for corrupt files, and file format limitations. Concatenating files from unknown or untrusted sources may even pose a security risk, and the process can be time-consuming, particularly for large files or multiple files.

How do I concatenate files in Linux and maintain the original line breaks?

To concatenate files in Linux while preserving the original line breaks, you can use the cat command along with the -s option, which replaces multiple blank lines with a single blank line. By default, the cat command removes blank lines between files when concatenating them, but the -s option preserves them. To concatenate two text files named “file1.txt” and “file2.txt” while preserving their original line breaks, use the command cat -s file1.txt file2.txt > combined.txt. This command will combine the contents of the two files while maintaining their original line breaks and write them into a new file called combined.txt. The -s option can also be used to remove blank lines from a single file by specifying the file name after the -s option.

How can I concatenate files in Linux while excluding duplicate lines?

The command “sort -u file1 file2 > newfile” will concatenate the contents of “file1” and “file2” and sort the combined content alphabetically, removing any duplicate lines. The resulting sorted and unique content will then be saved in a new file called “newfile“.
This command is useful when you have multiple files with related information that you want to merge and remove any duplicates. The -u option ensures that only unique lines are included in the output, which can be helpful in various data processing tasks.

Can I concatenate files in Linux from different directories using the cat command?

Yes, you can use the cat command to concatenate files in Linux from different directories. Simply specify the full path to each file when running the command, and the concatenated output will be saved to a new file. Your command should look like this:
cat /home/user/documents/file1.txt /home/user/downloads/file2.txt > combined.txt
However, ensure that you have the necessary permissions to access the files and directories before concatenating them.

Ojash

Author

Ojash is a skilled Linux expert and tech writer with over a decade of experience. He has extensive knowledge of Linux's file system, command-line interface, and software installations. Ojash is also an expert in shell scripting and automation, with experience in Bash, Python, and Perl. He has published numerous articles on Linux in various online publications, making him a valuable resource for both seasoned Linux users and beginners. Ojash is also an active member of the Linux community and participates in Linux forums.

Akshat

Reviewer

Akshat is a software engineer, product designer and the co-founder of Scrutify. He's an experienced Linux professional and the senior editor of this blog. He is also an open-source contributor to many projects on Github and has written several technical guides on Linux. Apart from that, he’s also actively sharing his ideas and tutorials on Medium and Attirer. As the editor of this blog, Akshat brings his wealth of knowledge and experience to provide readers with valuable insights and advice on a wide range of Linux-related topics.

Share this article
Shareable URL
Prev Post

10 Best Ways to Use Linux hexdump Command

Next Post

How to Check the None Value in Python [4 Best Ways]

Leave a Reply

Your email address will not be published. Required fields are marked *

Read next