DiffTools Developments

Lou D’Andria, Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

Last year, we discussed the newly-released DiffTools paclet from the Wolfram Language Paclet Repository. Since then, DiffTools has advanced significantly, and has been integrated directly into the Wolfram System starting with version 14.1. In this session, we’ll examine the evolution of the design, explore new diff-related features, and discuss those planned for future versions of the Wolfram Language.

Overview

The Wolfram/DiffTools paclet is still on the Paclet Repository and is still useable in 14.0.

Starting in 14.1, DiffTools is built in to the Wolfram Language, with some changes.

(HyperlinkPaclet["Diff", paclet:ref/Diff]) supports lists, strings, files, cells, notebooks, and more to come.

By supporting cells, (HyperlinkPaclet["Diff", paclet:ref/Diff]) effectively supports arbitrary expressions.

Returns a computable (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject]) with instant access to various views.

We even support 3-way diffs -- (HyperlinkPaclet["Diff3", paclet:ref/Diff3]) -- to analyze two different edits of the same list, string, …

This is very important in for source control systems.

Lists & DiffObject

(HyperlinkPaclet["Diff", paclet:ref/Diff]) builds on (HyperlinkPaclet["SequenceAlignment", paclet:ref/SequenceAlignment])

list1=Range[100];
list2=Range[5,110];

SequenceAlignment[list1, list2]

(HyperlinkPaclet["Diff", paclet:ref/Diff]) returns a (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject]), which displays as an interface

obj=Diff[list1, list2]

List available properties of a (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject])

obj["Properties"]

Get a property value

obj["SummaryString"]

List available formats of a (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject])

obj["Formats"]

Get a standalone view

obj["SideBySide"]

(HyperlinkPaclet["DiffApply", paclet:ref/DiffApply]) applies the changes represented by a (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject]) to an expression.

Strings & Text Files

String diffs provide different kinds of visualizations.

Diff["This sentnce has two errors.", "This sentence has no errors."]

DiffGranularity lets you consider a string word-by-word or line-by-line, rather than character-by-character.

Diff["This sentnce has two errors.", "This sentence has no errors.", DiffGranularity->"Words"]

Non-notebook files are typically handled as if their contents are strings, with granularity “Lines”.

Diff[File["DiffTools/Assets/Samples/Text1.txt"],File["DiffTools/Assets/Samples/Text2.txt"]]

Notebooks

{nbfile1,nbfile2}={,};

A notebook file contains a WL expression containing Cell expressions, grouping, options, and other metadata.

Diff[nbfile1,nbfile2]

Report view shows detailed information about the diff.

Options like (HyperlinkPaclet["DiffIgnore", paclet:ref/DiffIgnore]) let you customize the diff.

Diff[nbfile1, nbfile2,DiffIgnore -> "NotebookOptions"]

Use the Compare Notebooks palette to diff notebooks

“What did I do to this notebook since the last time I saved it?”

Diff[File @ NotebookFileName[], EvaluationNotebook[],"Report", DiffIgnore -> {"CellOptions" -> {CellLabel}}]

(HyperlinkPaclet["DiffApply", paclet:ref/DiffApply]) now works for notebooks!

Quiet @ DeleteFile[FileNameJoin[{$TemporaryDirectory,"DiffApplied.nb"}]];
DiffApply[
Diff[nbfile1,nbfile2], 
nbfile1, 
FileNameJoin[{$TemporaryDirectory,"DiffApplied.nb"}]]

NotebookOpen[%]

Cells & Expressions

Cells contents might be simple strings, or arbitrary expressions.

Expand[(a+b)^10+(b-c)^4]

cellobj1=PreviousCell[];

Expand[(a+b)^10-(a+c)^6]

cellobj2=PreviousCell[];

Diff[cellobj1, cellobj2]

Cell contents are handled as linear sequence of “tokens”.

Metadata like cell style and options are also part of the diff.

(HyperlinkPaclet["DiffInputFunction", paclet:ref/DiffInputFunction]) is a convenient way to turn expressions into cells.

Diff[Expand[(a+b)^10+(b-c)^4],Expand[(a+b)^10-(a+c)^6]]

Diff[Expand[(a+b)^10+(b-c)^4],Expand[(a+b)^10-(a+c)^6],DiffInputFunction->ExpressionCell]

No nesting support for in views yet.

Diff[∫(1/(1-(z^3)))\[DifferentialD]z,∫(1/(1+(z^3)))\[DifferentialD]z,DiffInputFunction->ExpressionCell]

Use the Compare Cells palette to diff cells in place.

Extensive DiffView changes

Biggest change: DiffView no longer exists!

All its functionality is available via new syntaxes for (HyperlinkPaclet["Diff", paclet:ref/Diff]) and (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject]).

obj=Diff[list1, list2]

obj["Formats"]

obj["UnifiedSummary"]

Diff[list1, list2,"UnifiedSummary"]

Diff[list1, list2,Appearance -> "UnifiedSummary"]

Notebook “Report” view is more highly styled.

Views are more consistent, beautiful, and available.

Grids and headings are more consistent and sensible.

Common views like “ArrayPlot” are now available for all diff types.

Diff[cellobj1,cellobj2, Appearance -> "ArrayPlot"]

Diff3[Range[50],Append[Range[50],"foo"],Append[Range[50],"bar"], Appearance -> "ArrayPlot"]

(HyperlinkPaclet["Diff3", paclet:ref/Diff3]) has many more view formats than before.

["Formats"]

We’ve added new formats, including accessing the raw data behind other views.

obj=Diff[File @ NotebookFileName[],EvaluationNotebook[]];
obj["Formats"]

{obj["ArrayPlot"], obj["ArrayPlotData"]}

Notable changes since 14.0

Function syntaxes have been streamlined and made more consistent throughout.

DiffView features available through  (HyperlinkPaclet["Diff", paclet:ref/Diff]) and (HyperlinkPaclet["DiffObject", paclet:ref/DiffObject]).

DiffView and DiffInvert were moved to an internal context.

(HyperlinkPaclet["DiffApply", paclet:ref/DiffApply]) now is supported for all relevant diff types.

applying a string diff to a text file

applying 3-way diffs, as long as they are conflict-free

(HyperlinkPaclet["DiffApply", paclet:ref/DiffApply]) now recognizes and correctly handles many different failure modes.

DiffApply[
Diff3[{1,2,3},{1,2,3,4},{1,2,3,5}],
{1,2,3}]

DiffApply[
Diff["This sentnce has two errors.", "This sentence has no errors."],
{1,2,3}]

DiffApply[
Diff[{1,2,3,4},{1,2,3}],
{1,2}]

Miscellaneous option name changes.

(HyperlinkPaclet[DiffAlignmentMethod, paclet:/ref/DiffAlignmentMethod])

(HyperlinkPaclet[DiffGranularity, paclet:/ref/DiffGranularity])

(HyperlinkPaclet[DiffIgnore, paclet:/ref/DiffIgnore])

(HyperlinkPaclet[DiffIncludeMatches, paclet:/ref/DiffIncludeMatches])

(HyperlinkPaclet[DiffInputFunction, paclet:/ref/DiffInputFunction])

(HyperlinkPaclet[DiffStyle, paclet:/ref/DiffStyle])

Efficiency

Remember, (HyperlinkPaclet["Diff", paclet:ref/Diff]) is built on (HyperlinkPaclet["SequenceAlignment", paclet:ref/SequenceAlignment]).

(HyperlinkPaclet["SequenceAlignment", paclet:ref/SequenceAlignment]) Method settings use different algorithms, resulting in different performance and different alignments.

(HyperlinkPaclet["\"Global\"", paclet:ref/SequenceAlignment#715743726]) (default): global Needleman–Wunsch alignment

(HyperlinkPaclet["\"Local\"", paclet:ref/SequenceAlignment#63743894]): local Smith–Waterman alignment

(HyperlinkPaclet["\"AlignByLongestSubsequences\"", paclet:ref/SequenceAlignment#1611445616]) (new in 14.1): divide-and-conquer heuristic approximation to aligning by the longest common (not necessarily contiguous) sequence

(HyperlinkPaclet["\"AlignByLongestCommonSequence\"", paclet:ref/SequenceAlignment#1094124650]) (new in 14.1): maximizes the number of characters or list elements common to both sequences

This can make a big difference for (HyperlinkPaclet["SequenceAlignment", paclet:ref/SequenceAlignment]), and thus for (HyperlinkPaclet["Diff", paclet:ref/Diff]).

AbsoluteTiming @ Diff[Range[10^4],Range[10^4+1]]

AbsoluteTiming @ Diff[Range[10^4],Range[10^4+1], Method -> "AlignByLongestCommonSequence"]

More comparisons.

diffMethodTimingGrid[Range[10^3],Range[10^3]//RandomSample]

diffMethodTimingGrid[Range[10^4],Range[10^4]//RandomSample]

diffMethodTimingGrid[StringRiffle[RandomWord[10^3], " "], StringRiffle[RandomWord[10^3], " "]]

What’s next?

(HyperlinkPaclet["DiffIgnore", paclet:ref/DiffIgnore]) improvements, including whitespace awareness

Support conflict resolutions for 3-way diffs

Diff/merge integration with command-line diffs

More improvements to (HyperlinkPaclet["DiffApply", paclet:ref/DiffApply]) to support context-sensitive patching

Richer unified report view for notebooks

Lots more…