nsqd: consider publicly exporting diskQueue

[jaytaylor]

First off, thank you for DiskQueue! It's the best go implementation of a disk-backed channel-esque data structure that I've found!

I have been using it and it's great, but I'm sure you can imagine that hand-applying updates sucks!

Would you please consider making diskQueue publicly exported so it can be used as a "normal" dependency?

In case it is helpful, here is a copy of the ported DiskQueue I'm currently using (obviously there is a lot more stuff I ported over, including interfaces and unit-tests, etc):

https://gist.github.com/jaytaylor/71ae326116d9d5492250022025a2015a

Also I am curious - is there a way to get the disk queue to delete or clear out the .dat files? I've invoked .Empty() which reset the metadata, but the .dat file lives on at the same size as before (even though Depth()==0).

[mreiferson]

yea, we probably should

[ploxiln]

If there is a chance of #625 (replace diskQueue with a write-ahead log) eventually getting done, it would be kinda funny to have exported the original diskQueue

[jaytaylor]

Can we get it moved out to it's own repository then, perhaps?

[stephensearles]

I don't think #625 should conflict with this. I could be wrong here, but the interface, I'd guess, will stay mostly the same? It's more like, with #625, that diskQueue will be used a little differently by NSQ to promptly store things and give clients more assurance about that, but it shouldn't fundamentally change writing a stream of events to disk.

[jaytaylor]

What are your thoughts on possibly relocating DiskQueue into it's own external repository?

[mreiferson]

That would seem to be the likely path if we do export it.

[didip]

Count me in as one other person that's been forking diskqueue.go and applying upstream patch by hand.

It would be super awesome if DiskQueue is publicly exportable.

[mreiferson]

OK, fine

[mreiferson]

see #847

[jaytaylor]

@mreiferson Thank you!