Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer-backed access (full implementation) #299

Merged
merged 9 commits into from
Jun 10, 2021

Conversation

mkitti
Copy link
Member

@mkitti mkitti commented Nov 17, 2020

Buffer-backed access (full implementation)

These changes complete #278 and fixes #5

7cc0468 implements a lightweight interface DataAccess with a createView(Object) method as per the Twitter discussion. It also includes a default implementation that just returns this. The interface definition returns type Object, which seems better than my attempt to parameterize generics across the entire Access tree.

6f851db contains the BufferAccess interface and its implementations. Per @hanslovsky's design in #278 BufferAccess implements ArrayDataAccess and VolatileAccess. The concrete classes share a common AbstractBufferAccess.

The net.imglib2.img.basictypeaccess.nio tree is as follows:

  • BufferAccess implements VolatileAccess, ArrayDataAccess
    • AbstractBufferAccess
      • ByteBufferAccess
      • CharBufferAccess
      • FloatBufferAccess
      • DoubleBufferAccess
      • IntBufferAccess
      • ShortBufferAccess
      • LongBufferAccess

Answers to open questions in #278

General

  1. ArrayDataAccess is implemented as follows
    • createArray( int numEntities ) allocates a new java.nio.Buffer based on whether the current one isDirect or not and if the current Access isValid
    • getCurrentStorageArray() returns the raw Buffer without duplicating it
    • getArrayLength() is returns Buffer.limit() (as opposed to Buffer.capacity() as in Buffer-backed accesses #278)
  2. VolatileAccess is implemented with support from AbstractBufferAccess
  3. All concrete classes store a type-specific Buffer

Specific

  1. As in Buffer-backed accesses #278 (comment) and item 1 above, createArray( int numEntities ) allocates a Buffer that has the same directness as the current Buffer

Differences to imglib[1] NIO*Array

The current implementation differs from the previous NIO*Array (e.g. NIOLongArray :

  1. The constructor receives a java.nio.Buffer directly rather than array. The user can wrap (e.g. LongBuffer.wrap) an array if needed.
  2. getCurrentStorageArray returns a BufferAccess rather than an Array

New Items

  1. I named the classes LongBufferAccess rather than NIOLongArray or LongBufferLongAccess. The buffers are distinct from arrays. Also the Buffer type and Access type are always the same in this implementation.
  2. I proxied in the bulk relative get and put methods of java.nio.LongBuffer as getValues and setValues that allows the user to quickly copy in or out data using the corresponding Array access (e.g. AbstractLongArray as of f5609a6). This is mainly a convenience and could be dropped.
  3. Buffer.duplicate() is only used in DataAccess.createView(Object) and in the bulk getValues and setValues methods. Buffer.duplicate() only actually exists in Java 9+ but is implemented in the concrete classes in Java 8 (e.g. LongBuffer)

Change Log

  • f5609a6 Changed to using the Abstract*Array types rather than *Array.

@mkitti
Copy link
Member Author

mkitti commented Nov 17, 2020

I went a bit further to parameterize DataAccess and other subinterfaces, but this seemed too much and is not included in this pull request.
mkitti@5bd41a2

@hanslovsky
Copy link
Member

Beautiful PR! I will have a look, baby boy permitting

@mkitti
Copy link
Member Author

mkitti commented Nov 17, 2020

Thanks @hanslovsky . I really appreciated your initial notes on this issue.

@tpietzsch tpietzsch self-requested a review November 18, 2020 08:29
@hanslovsky
Copy link
Member

I left a few comments. Overall, this looks good to me. My only real concern is that the buffer type is hard-coded to ByteBuffer in AbstractBufferAccess.allocate, cf #299 (comment)

I will leave the more thorough review to the active maintainers.

Change NUM_BYTES to NUM_BYTES_PER_ENTITY along with getter
Make DEFAULT_IS_VALID final
Allow longer type-specific Buffers
Copy link
Member

@hanslovsky hanslovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proposed changes look good to me but I will leave further review/merging to the active maintainers

@tpietzsch
Copy link
Member

I pushed some changes to https://github.com/imglib/imglib2/commits/nio_buffers:

  • Remove unnecessary restriction to A extends DataAccess from Cell<A>.
  • Changed protected to package-private for classes in net.imglib2.img.basictypeaccess.nio. Please reinstate protected if you need to derive additional classes outside imglib2 core.
  • additional formatting.

If you agree with these changes please merge into your PR.
Otherwise this looks good to go.

I'm not sure whether the replacement of A by A extends DataAccess breaks binary compatibility (but I don't think so)?

@mkitti
Copy link
Member Author

mkitti commented Nov 25, 2020

On binary compatibility, I may have come across some examples where I had very generic code break because A was previously unconstrained and the method signatures no longer matched. This may be an extreme case, however.

@tpietzsch
Copy link
Member

On binary compatibility, I may have come across some examples where I had very generic code break because A was previously unconstrained and the method signatures no longer matched. This may be an extreme case, however.

Yes, that is definitely a problem. It happened to us before, e.g. #280

Also here, the erasure of A changes from Object to DataAccess.
In this case we might be safe because the A only appears as a type parameter in e.g. ArrayImg<T,A> and that is erased to ArrayImg no matter what. But maybe I'm overlooking something unrelated to the changed classes.

@mkitti
Copy link
Member Author

mkitti commented Nov 25, 2020

I went back a few commits, and I think the issue I encountered was more due to having multiple versions of imglib2 on my path at the time.

I think this might have been the code that caused a NoSuchMethodError . Might there be a binary compatibility issue with the constructor of ArrayImg?

	public static < T extends NativeType< T >, A extends DataAccess > ArrayImg< T, A > createArrayImg(
			final NativeTypeFactory< T, A > factory, final A data, final long... dims)
        {
		final Fraction fraction = factory.createLinkedType(null).getEntitiesPerPixel();
		final ArrayImg< T, A > img = new ArrayImg< T, A >(data, dims, fraction);
		final T linkedType = factory.createLinkedType(img);
		img.setLinkedType(linkedType);
		return img;
	}

@mkitti
Copy link
Member Author

mkitti commented Dec 3, 2020

After reading more about binary compatibility, it seems unwise to add a bound to A. Rather we should check if A is a DataAccess and invoke createView only then:

public class ArrayImg< T extends NativeType< T >, A > extends AbstractNativeImg< T, A > implements SubIntervalIterable< T >
...
	@SuppressWarnings("unchecked")
	@Override
	public A update( final Object o )
	{
		if(data instanceof DataAccess)
			return ( A ) ( (DataAccess) data).createView( o );
		else
			return data;
	}

https://docs.oracle.com/javase/specs/jls/se8/html/jls-13.html#jls-13.4.13

13.4.13. Method and Constructor Type Parameters
Adding or removing a type parameter of a method or constructor does not, in itself, have any implications for binary compatibility.

If such a type parameter is used in the type of the method or constructor, that may have the normal implications of changing the aforementioned type.

Renaming a type parameter of a method or constructor has no effect with respect to pre-existing binaries.

Changing the first bound of a type parameter of a method or constructor may change the erasure (§4.6) of any member that uses that type parameter in its own type, and this may affect binary compatibility. Specifically:

If the type parameter is used as the type of a field, the effect is as if the field was removed and a field with the same name, whose type is the new erasure of the type variable, was added.

If the type parameter is used as the type of any formal parameter of a method, but not as the return type, the effect is as if that method were removed, and replaced with a new method that is identical except for the types of the aforementioned formal parameters, which now have the new erasure of the type parameter as their type.

If the type parameter is used as a return type of a method, but not as the type of any formal parameter of the method, the effect is as if that method were removed, and replaced with a new method that is identical except for the return type, which is now the new erasure of the type parameter.

If the type parameter is used as a return type of a method and as the type of one or more formal parameters of the method, the effect is as if that method were removed, and replaced with a new method that is identical except for the return type, which is now the new erasure of the type parameter, and except for the types of the aforementioned formal parameters, which now have the new erasure of the type parameter as their types.

Changing any other bound has no effect on binary compatibility.

@mkitti
Copy link
Member Author

mkitti commented Dec 6, 2020

I've removed bounds on A so there are no longer binary compatibility issues. Type erasure should just make A an Object in most cases.

@tpietzsch
Copy link
Member

Hmm, having an instanceof check per invocation may be a slight performance problem. ArrayImg only does it once per accessor, but for example CellImg does it everytime an accessor moves into a new cell, which can be relatively frequently. I would prefer minor API breakage over this, or benchmarks that show that there is no performance impact.

@mkitti
Copy link
Member Author

mkitti commented Dec 7, 2020

I moved the instanceof checks into the constructor.

In ArrayImg I just created an anonymous DataAccess to wrap around an arbitrary object if it is not a DataAccess. One consequence of this design is that DataAccess.createView now just returns an Object. If this needs to be constrained at some point, a parameterized sub-interface might make sense in the future.

For AbstractCellImg, I use a BiFunction in AbstractCellImg.createView set at the constructor. On first execution, it checks if A is a DataAccess and then sets AbstractCellImg.createView either to use DataAccess.createView or to just return data. [edited for 996fdec]

Let me know if you think these measures are worth saving API breakage for or not.

@mkitti
Copy link
Member Author

mkitti commented Dec 7, 2020

I do not see a major performance regression when running the tests.

PS>git branch --show-current
master
PS>git rev-parse HEAD
d54bf15cb17d5a22599c687474c267efa573e472
PS>java -jar '..\..\junit-platform-console-standalone-1.7.0.jar' -cp $env:CLASSPATH -p net.imglib2.img.cell

Thanks for using JUnit! Support its development at https://junit.org/sponsoring

.
+-- JUnit Jupiter [OK]
'-- JUnit Vintage [OK]
  +-- CellContainerFactoryTest [OK]
  | +-- testBitDefaultCellDimensions [OK]
  | +-- testFloatDefaultCellSize [OK]
  | +-- testIntDefaultCellDimensions [OK]
  | +-- testFloatDefaultCellDimensions [OK]
  | +-- testIntDefaultCellSize [OK]
  | '-- testBitDefaultCellSize [OK]
  +-- CellImgTest [OK]
  | +-- testCellImgInvalidDimensions [OK]
  | '-- testCellImg [OK]
  +-- CellTest [OK]
  | +-- testLocalIndexCalculation [OK]
  | '-- testGlobalPositionCalculation [OK]
  +-- CopyTest [OK]
  | +-- testCopyToArrayContainerWithDestIteration [OK]
  | +-- testCopyToCellContainerWithDestIteration [OK]
  | +-- testCopyArrayToArrayWithIterationBoth [OK]
  | +-- testCopyToCellContainerWithSourceIteration [OK]
  | '-- testCopyToArrayContainerWithSourceIteration [OK]
  +-- CellContainerTest [OK]
  | '-- equalIterationOrder [OK]
  '-- CellCursorTest [OK]
    +-- testResetWithCursor [OK]
    +-- testSumWithLocalizingCursor [OK]
    +-- testResetWithLocalizingCursor [OK]
    +-- testSumWithRandomAccess [OK]
    +-- testJmpWithLocalizingCursor [OK]
    +-- testSumWithCursor [OK]
    '-- testJmpWithCursor [OK]

Test run finished after 2171 ms
[         8 containers found      ]
[         0 containers skipped    ]
[         8 containers started    ]
[         0 containers aborted    ]
[         8 containers successful ]
[         0 containers failed     ]
[        23 tests found           ]
[         0 tests skipped         ]
[        23 tests started         ]
[         0 tests aborted         ]
[        23 tests successful      ]
[         0 tests failed          ]

PS>git checkout nio_buffers
Switched to branch 'nio_buffers'
PS>git rev-parse HEAD
996fdec583000669d5a8d223c560bcd45ae43ccf

[... rebuild ...]

PS>java -jar '..\..\junit-platform-console-standalone-1.7.0.jar' -cp $env:CLASSPATH -p net.imglib2.img.cell

Thanks for using JUnit! Support its development at https://junit.org/sponsoring

.
+-- JUnit Jupiter [OK]
'-- JUnit Vintage [OK]
  +-- CellContainerFactoryTest [OK]
  | +-- testBitDefaultCellDimensions [OK]
  | +-- testFloatDefaultCellSize [OK]
  | +-- testIntDefaultCellDimensions [OK]
  | +-- testFloatDefaultCellDimensions [OK]
  | +-- testIntDefaultCellSize [OK]
  | '-- testBitDefaultCellSize [OK]
  +-- CellImgTest [OK]
  | +-- testCellImgInvalidDimensions [OK]
  | '-- testCellImg [OK]
  +-- CellTest [OK]
  | +-- testLocalIndexCalculation [OK]
  | '-- testGlobalPositionCalculation [OK]
  +-- CopyTest [OK]
  | +-- testCopyToArrayContainerWithDestIteration [OK]
  | +-- testCopyToCellContainerWithDestIteration [OK]
  | +-- testCopyArrayToArrayWithIterationBoth [OK]
  | +-- testCopyToCellContainerWithSourceIteration [OK]
  | '-- testCopyToArrayContainerWithSourceIteration [OK]
  +-- CellContainerTest [OK]
  | '-- equalIterationOrder [OK]
  '-- CellCursorTest [OK]
    +-- testResetWithCursor [OK]
    +-- testSumWithLocalizingCursor [OK]
    +-- testResetWithLocalizingCursor [OK]
    +-- testSumWithRandomAccess [OK]
    +-- testJmpWithLocalizingCursor [OK]
    +-- testSumWithCursor [OK]
    '-- testJmpWithCursor [OK]

Test run finished after 2177 ms
[         8 containers found      ]
[         0 containers skipped    ]
[         8 containers started    ]
[         0 containers aborted    ]
[         8 containers successful ]
[         0 containers failed     ]
[        23 tests found           ]
[         0 tests skipped         ]
[        23 tests started         ]
[         0 tests aborted         ]
[        23 tests successful      ]
[         0 tests failed          ]

@tpietzsch
Copy link
Member

Let me know if you think these measures are worth saving API breakage for or not.

Without these latest changes the code is more understandable. I vote for API breakage. I think the fallout downstream will be minor if at all.

@axtimwalde @StephanPreibisch what do you think?

@mkitti
Copy link
Member Author

mkitti commented Dec 8, 2020

To summarize,

  1. When using NativeImg.update we need to return a view of the underlying java.nio.Buffer since the Buffer classes contain state which is problematic for thread safety.
  2. We implemented a DataAccess interface with a default method createView that just returns this.
  3. Implementations of NativeImg.update would now return the result of DataAccess.createView
  4. Most of the access classes in imglib2 now implement DataAccess.
  5. The simple implementation is add a bound to the type parameter A of ArrayImg and AbstractCellImg so that A extends DataAccess.
  6. The issue is that this changes the method signatures since type erasure of A results in DataAccess rather than Object potentially breaking binary compatibility.
  7. The resulting code, however, is simple.
    master...mkitti:nio_buffers_simple

@mkitti
Copy link
Member Author

mkitti commented Dec 8, 2020

If we do not want to break binary compatibility, then the solution is

  1. Do not bound A in ArrayImg and AbstractCellImg so method and constructor signatures do not change.
  2. Check if A instanceof DataAccess in the ArrayImg or AbstractArrayImg constructor.
  3. Provide distinct implementations of NativeImg.update depending on whether A instanceof DataAccess or not.

@axtimwalde
Copy link
Member

I prefer breaking API over instanceof.

@mkitti
Copy link
Member Author

mkitti commented Dec 9, 2020

OK, I forced pushed back to c6a9d07 by @tpietzsch .

Might alternate constructors like the following help binary compatibility?

	@SuppressWarnings( "unchecked" )
        @Deprecated
	public ArrayImg( final Object data, final long[] dim, final Fraction entitiesPerPixel ) {
		this( ( A ) data, dim, entitiesPerPixel );
	}
	@SuppressWarnings( "unchecked" )
	@Deprecated
	public ArrayImgAWTScreenImage( final T type, final Object data, final long[] dim ) {
		this( type, (A) data, dim );
	}

Let me see if I can write a test case.

@tpietzsch
Copy link
Member

Might alternate constructors like the following help binary compatibility?

that's a great idea

A was previously unbounded, but is now bounded by the new interface
DataAccess
@mkitti
Copy link
Member Author

mkitti commented Dec 15, 2020

After adding the additional constructors, there are no longer any binary compatibility issues. For the update method, Java generates two signatures, one of which still returns Object.

I used the following code to test binary compatibility:

		ShortAccess data = new ShortArray(new short[] { 1, 2, 3, 4, 5});
		
		long[] dim = new long[] { 5 };
		ArrayImg< UnsignedShortType, ShortAccess > img = new ArrayImg<>(data, dim, new Fraction());
		UnsignedShortType type = new UnsignedShortType( img );
		img.setLinkedType(type);
		
		ShortAccess updatedData = img.update(this);
		System.out.println(updatedData);

I put this in the Sandbox class from the imglib2-tutorials. Sandbox should be compiled against imglib2 via the scijava parent pom. I then I ran it directly from the command line with distinct exported jars of imglib2 from Eclipse.

Without adding the constructors taking Object as a parameter in lieu of A, I get the following error:

Exception in thread "main" java.lang.NoSuchMethodError: net.imglib2.img.array.ArrayImg.<init>(Ljava/lang/Object;[JLnet/imglib2/util/Fraction;)V

Adding the constructors, everything seems to work fine. I get the following as output.

net.imglib2.img.basictypeaccess.array.ShortArray@668bc3d5

I expected to encounter issues with the update method, so I used javap -c Sandbox.class to see the bytecode:

      75: invokevirtual #30                 // Method net/imglib2/img/array/ArrayImg.setLinkedType:(Lnet/imglib2/type/NativeType;)V
      78: aload_3
      79: aload_0
      80: invokevirtual #34                 // Method net/imglib2/img/array/ArrayImg.update:(Ljava/lang/Object;)Ljava/lang/Object;
      83: checkcast     #38                 // class net/imglib2/img/basictypeaccess/ShortAccess

Sandbox is calling update with Object parameter and an Object return type.

I then ran javap -s ArrayImg.class and discovered the following:

Compiled from "ArrayImg.java"
public class net.imglib2.img.array.ArrayImg<T extends net.imglib2.type.NativeType<T>, A extends net.imglib2.img.basictypeaccess.DataAccess> extends net.imglib2.img.AbstractNativeImg<T, A> implements net.imglib2.view.iteration.SubIntervalIterable<T> {
...
  public A update(java.lang.Object);
    descriptor: (Ljava/lang/Object;)Lnet/imglib2/img/basictypeaccess/DataAccess;
...
  public java.lang.Object update(java.lang.Object);
    descriptor: (Ljava/lang/Object;)Ljava/lang/Object;
}

ArrayImg contains two distinct update method signatures differing only in return type! This is because ArrayImg implements the interface NativeImg which after type erasure specifies an update method that returns Object. Java implements co-variant return types by including both method signatures.

My conclusion is that with the added constructors in cacce9f, there are no longer binary compatibility issues with this pull request.

@imagesc-bot
Copy link

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/imariswriter-open-source-library-for-storage-of-large-images-in-blockwise-multi-resolution-format/41997/12

@hanslovsky
Copy link
Member

@tpietzsch Do you think this can be merged?

@axtimwalde
Copy link
Member

We should merge this and fix problems as they emerge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Port NIO-backed images to ImgLib2
5 participants