PowerShell Script to Split a Large Text File into Smaller Files

PowerShell script to split large text files into smaller files by line count, efficiently handling big data without high memory usage and creating ordered outpu

Brief Introduction

PowerShell Script to Split a Large Text File into Smaller Files (powershell)
DESCRIPTION:
This PowerShell script allows splitting a large text file into multiple smaller files by specifying the maximum number of lines per output file. Useful for handling log files or data too large to be easily opened or processed.

ADDITIONAL NOTES:
The script reads the text file line by line to avoid loading the entire file into memory, making it efficient even with very large files. It creates sequentially numbered output files in the specified folder.

EXPLANATIONS:
- Uses Get-Content with the -ReadCount 1 option to read one line at a time, reducing memory usage.
- The line and file counters help determine when to create a new output file.
- The output file names include a progressive index to maintain order.
- The creation of the output folder is handled if it does not exist.

Example Code: powershell

# Script to split a large text file into smaller files
param(
    [Parameter(Mandatory=$true)]
    [string]$FileInputPath,          # Full path of the text file to split

    [Parameter(Mandatory=$true)]
    [int]$MaxRighePerFile,           # Maximum number of lines per output file

    [Parameter(Mandatory=$false)]
    [string]$CartellaOutput = "Output" # Folder to save the split files (default: ./Output)
)

# Check that the input file exists
if (-not (Test-Path -Path $FileInputPath)) {
    Write-Error "The input file '$FileInputPath' does not exist."
    exit 1
}

# Create the output folder if it does not exist
if (-not (Test-Path -Path $CartellaOutput)) {
    New-Item -ItemType Directory -Path $CartellaOutput | Out-Null
}

# Extract the base file name without extension
$NomeBaseFile = [System.IO.Path]::GetFileNameWithoutExtension($FileInputPath)

# Extract the file extension
$EstensioneFile = [System.IO.Path]::GetExtension($FileInputPath)

# Initialize counters
$ContatoreFile = 1
$ContatoreRighe = 0

# Build the output file name
$NomeFileOutput = "{0}_{1}{2}" -f $NomeBaseFile, $ContatoreFile, $EstensioneFile
$PercorsoFileOutput = Join-Path -Path $CartellaOutput -ChildPath $NomeFileOutput

# Open the output file for writing
$StreamWriter = [System.IO.StreamWriter]::new($PercorsoFileOutput, $false)

try {
    # Read the input file line by line
    Get-Content -Path $FileInputPath -ReadCount 1 | ForEach-Object {
        $riga = $_

        # Write the line to the current output file
        $StreamWriter.WriteLine($riga)
        $ContatoreRighe++

        # If the maximum number of lines is reached, close the file and open a new one
        if ($ContatoreRighe -ge $MaxRighePerFile) {
            $StreamWriter.Close()
            $ContatoreFile++
            $ContatoreRighe = 0

            $NomeFileOutput = "{0}_{1}{2}" -f $NomeBaseFile, $ContatoreFile, $EstensioneFile
            $PercorsoFileOutput = Join-Path -Path $CartellaOutput -ChildPath $NomeFileOutput
            $StreamWriter = [System.IO.StreamWriter]::new($PercorsoFileOutput, $false)
        }
    }
}
finally {
    # Ensure the stream is closed even in case of error
    if ($StreamWriter -ne $null) {
        $StreamWriter.Close()
    }
}

Write-Host "Splitting completed. Files created in folder: $CartellaOutput"
×
×
×
Do you want to regenerate the answer?
×
Export chat
Choose export format
Generazione PDF in corso…
Allegati
×
You are about to close the chat and switch to the image generator. If you are not logged in, you will lose our chat. Do you confirm?
You are viewing a shared chat in temporary mode. It will not be saved.
×
Saved prompts
×
Private note
×
Label
×
Search all chats
×
Your insights
Analyzing…
×
AI Preferences
Assistant name 30
Answer length
AI model
Structured answers
Include examples
Use emojis
Guide me, don't solve Hints and questions instead of a ready answer
More settings →
×
Share this chat
Anyone opening this link can view the chat or add it to their profile as their own chat.
Note: the chat attachments will be shared too. Anyone who adds it will receive a copy of the files in their own folder.
Shared chat
Someone shared a chat with you. Do you want to just view it or add it to your chats?
Note: the chat attachments will be shared too. Anyone who adds it will receive a copy of the files in their own folder.
×

Saved messages

Loading...

×

Chat History

example-programs · CHAT HISTORY

Loading...

AI Preferences

×
  • BasicQuick and essential answers for study
  • MediumHigher quality for study and programming
  • AdvancedComplex reasoning and detailed analysis
Explain Steps

What do you need?

PowerShell 5.1 or higher

Input text file (.txt)

DEPENDENCIES: no external, uses only standard PowerShell cmdlets

CONFIG: input parameters - source file path, maximum number of lines per output file, destination folder for split files

Similar Topics

PowerShell Utility to Generate Simple Random Passwords
PowerShell script to generate simple random passwords with user-defined length using alphanumeric characters for secure and easy password creation.
PowerShell Script to Rename Files Based on Creation Date
PowerShell script to rename files by adding creation date prefix in YYYYMMDD format for organized file management and conflict-free renaming.
PowerShell Utility to Search Files by Extension in a Folder
PowerShell script to search files by extension in a folder with full path output. Simple, reusable, and error-handled for efficient file searching.
PowerShell Script to List Running Processes and Save to CSV
PowerShell script to list running processes with details and save them to a CSV file for easy monitoring and analysis using built-in commands.
PowerShell Script to Count the Number of Files in Each Subfolder
PowerShell script to count files in each subfolder of a specified directory with clear output and error handling for non existing folders.
PowerShell Utility to Replace a String in All Text Files in a Folder
PowerShell script to search and replace strings in all text files within a folder, automating bulk text edits with error handling and no extra software.
PowerShell Script to Create a Default Folder Structure for a Project
PowerShell script to create a default folder structure for software projects, standardizing organization and speeding up setup on Windows systems.
PowerShell Script to Monitor Free Space on a Local Drive
PowerShell script to monitor free space on a local drive with customizable threshold and detailed console output for Windows PowerShell 5.1 or higher.