Recursive AI

Detect paragraph splits and merges in C#

OneDiff is a free, open-source diff library for .NET from Recursive AI. MIT or Apache-2.0, no native dependencies.

Find where paragraph breaks were added or removed in plain text, not DOCX. For digital PDFs, use RecursiveAI.OneDiff.Pdf 1.0.1. For DOCX, build a CompareDocument with your own reader; no released OneDiff package reads DOCX.

Terminal
dotnet add package RecursiveAI.OneDiff.Documents

Requires .NET 10 (net10.0); .NET 9 and earlier aren’t supported.

C#
using OneDiff.Documents;
using OneDiff.Documents.Model;
using OneDiff.Documents.Pipeline;
 
const string first =
    "Pack the orders before noon.";
const string second =
    "Ship every parcel on Monday.";
var one = CompareDocument.FromText(
    $"{first} {second}");
var two = CompareDocument.FromText(
    $"{first}\n\n{second}");
 
Show(one, two);
Show(two, one);
 
static void Show(CompareDocument before,
                 CompareDocument after)
{
    var settings = new CompareSettings
    {
        DetectSplitsAndMerges = true,
        SplitMergeThreshold = 0.8
    };
    var alignment = Stages.Align(
        before, after, settings: settings);
    foreach (var link in alignment.Links)
    {
        if (link.Kind != LinkKind.Split &&
            link.Kind != LinkKind.Merge)
            continue;
        int oldCount = link.OldNodes.Length;
        int newCount = link.NewNodes.Length;
        Console.WriteLine(
            $"{link.Kind}: " +
            $"{oldCount} → {newCount}");
    }
}

dotnet run prints:

Text
Split: 1 → 2
Merge: 2 → 1

OldNodes and NewNodes list the corresponding paragraph IDs. If the candidates are too short or dissimilar to form a link, the engine can report separate insertions or deletions instead.

Unzip the example, then dotnet run.

Variations

To see the effect of turning detection off, replace the settings inside Show with:

C#
var settings = new CompareSettings
{
    DetectSplitsAndMerges = false
};

The program prints nothing in this variation: it only prints split and merge links.

The short answer

Pass both versions to Stages.Align with DetectSplitsAndMerges enabled. Read the returned Links: a Split has one old node and several new ones; a Merge has several old nodes and one new one. Only the break between the sentences changes.

What is a line break vs. paragraph break?

A paragraph break ends one block and starts another. A line break can start a new line without ending the paragraph. In the sample, \n\n separates the two paragraphs passed to CompareDocument.FromText. Don’t add that separator at every wrap in text copied from a narrow column, or those wraps will become paragraph boundaries in the input. If your source already identifies paragraphs, keep those blocks instead of reconstructing them from the displayed line lengths.

What does the similarity threshold control?

Lower SplitMergeThreshold to catch more splits where the text was also edited; raise it to reduce false matches. The threshold is the minimum similarity for the whole block against a run of its candidate parts. The default is 0.8. It’s not a cutoff for comparing each short part to the full paragraph independently, and satisfying it isn’t a guarantee. The alignment rules require parts to be neighboring siblings, check windows of two to eight parts, and require the winning window to score more than 0.05 above the best single candidate. A candidate part also needs at least three words and enough shared word sequences with the longer block. Lowering the threshold does not remove those checks.

How is a split different from a move?

A move links one old block to one new block in a different position. A split or merge changes the number of blocks involved. Read the link kind and both node counts rather than guessing from how many characters were inserted or removed. A new paragraph boundary can be the only change even when the split touches a long passage.

All onediff recipes