It takes a while to dump a 161Mb file from ADFS.

[fayep]

Perhaps use mmap? I'm dumping the Domesday discs to use with beeblink. I don't think I actually need this giant file but honestly a modern PC with SSD should be dumping this in seconds, not minutes. The ADF is stored on a network share, so it could be latency...

[tom-seddon]

Thanks for the report! (I assume I must admit, I never tried adf_extract with anything larger than an ADFS L disk, so I'm relieved to hear it works at all.)

I agree that it should be quicker than "minutes"! Are these disk image files available for download anywhere?

The actual reading of the image hopefully won't be the problem, as that happens up front as a single read - but who knows. But I can see a couple of possible things now I"m looking at the code suspiciously.

[fayep]

The source files are on archive.org, but they're huge.
https://drive.google.com/file/d/1x9ctD10gZZxOyw0itvKagJ4Z_gjqWCjK/view?usp=share_link
This is chdman (MAME) compressed, you can extract it, but it's just the extracted data component of the analog disc.

[fayep]

https://archive.org/details/BBC-Domesday-AIV-LaserDisc-DD86-Disc-Set-4 archive.org link

[tom-seddon]

I unpacked it using chdman:

>C:\temp\mame0272b_64bit\chdman.exe extractraw -i c:\Users\tom\Downloads\Domesday_NationalA_SideA.chd -o c:\temp\Domesday_NationalA_SideA.raw
chdman - MAME Compressed Hunks of Data (CHD) manager 0.272 (mame0272)
Output File:  c:\temp\Domesday_NationalA_SideA.raw
Input CHD:    c:\Users\tom\Downloads\Domesday_NationalA_SideA.chd
Extraction complete

THis produced a 230,281,216 byte file, sha1sum 6da2bb8b2ccf686d0ffeda9b9b72c0c5b3fac17e.

Running it through adf_extract, it seems this is too short:

>adf_extract Domesday_NationalA_SideA.raw -o test
FATAL: bad ADFS format - disk image size mismatch (data is 899536 sectors (230281216 bytes), image says 1295784 sectors (331720704 bytes))

So the file DATA2 runs off the end.

I've tidied things up a bit so adf_extract can deal with this a bit better: there's a new --ignore-size-mismatch option. The terrible output speed is also much improved. There's still some more to be eked out but it extracts the disk in less than 3 seconds on my 10 year old laptop, which feels a bit more like.

f1a4042

[fayep]

Hey, this is amazing. Thanks for the fixes and the speed of working on them. The reason the file size is smaller is because the extractor only extends the file if there are new sectors extracted and if there is no data (eg. Audio is being stored) then it doesn't write more to the file - the size allows it to seek to these parts of the disk though so you can still access photos and video. I used truncate -s <size given by adf_extract> which creates a sparse extension to the file - basically free - and that's how I got through to the copy.. Sparse would be nice BTW, if you got 4k of 00 in a block, seek rather than writing. My next step is to look at beeblink and see if I can make the BeebSCSI open source ROM talk to Beeblink. I'm on stardot as Beiriannydd, so if you want to keep up with my crazy tinkering, you can friend me there.

[fayep]

BTW, I had to edit all the .inf files to add $. to the front before they'd work with beeblink.. I tried loading the !BOOT on B2 and it got very upset. I didn't expect that, but of course it's lacking the SCSI and AIV roms

[tom-seddon]

Yes, the .inf files produced by adf_extract aren't currently suitable for use with BeebLink directly - it's set up for some future BeebLink expansion that'll deal with hierarchical file structure. (This is something that's been on my todo list for years, and I will get round to it eventually.) I think they might also work with Disk Image Manager.

I don't know anything about the BBC's SCSI interface yet, though it's next on my list for b2, so I'll watch with interest!

The BeebLink ROM does have an OSWORD call for making server requests (https://github.com/tom-seddon/beeblink/blob/master/docs/tech.md#new-osword-call), though all the request types supported by the server are currently file-based. It could be extended with some block-based mechanism (fetching sectors from some set of specific disk image files) if that'd simplify things though.