To address the situation, I would essentially have to stop the SQL Server Agent job called 'Distribution clean up: distribution'. Oddly enough, subsequent runs would be right back on track.
To mitigate the issue as quickly as possible (perhaps there are other solutions out there) I decided to put together a simple monitoring job to check how long the cleanup job has been running. If it has been running for more than 10 minutes, stop the job.
Here is the code I used. It has to run in msdb and I scheduled it to run every 10 seconds. You could use this syntax to monitor any jobs - perhaps an index maintenance job is running abnormally long on the weekend and bloating TempDB or something - but you wouldn't know right away. This will at least stop the activity until there is time to further investigate.
DECLARE @Duration_secs INT;
SELECT @Duration_secs = DATEDIFF(ss, ja.start_execution_date, GETDATE())
FROM msdb.dbo.sysjobs jobs
LEFT JOIN msdb.dbo.sysjobactivity ja ON ja.job_id = jobs.job_id
AND ja.start_execution_date IS NOT NULL
WHERE jobs.name = 'Distribution clean up: distribution'
AND stop_execution_date IS NULL;
IF @Duration_secs > 600 -- 10 minutes
EXEC dbo.sp_stop_job N'Distribution clean up: distribution';