Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
Description
The unsafe off-heap in FLINK-13985 was implemented by hacking into a private constructor of java.nio.DirectByteBuffer. This causes undesirable warnings in Java 11:
WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.flink.core.memory.MemoryUtils (file:/C:/Development/repos/flink/flink-core/target/classes/) to constructor java.nio.DirectByteBuffer(long,int) WARNING: Please consider reporting this to the maintainers of org.apache.flink.core.memory.MemoryUtils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release
We should look into this, confirm that it is harmless and try to silence it to avoid confusion for users.
Attachments
Issue Links
- is caused by
-
FLINK-13985 Use unsafe memory for managed memory.
- Resolved
- relates to
-
FLINK-17524 Remove "An illegal reflective access operation has occurred" warnings on JDK11
- Open
- links to
Activity
ASF GitHub Bot
logged work - 25/Feb/20 09:31
-
- Time Spent:
- 10m
-
azagrebin commented on pull request #11160: [
FLINK-15094] Use Unsafe to instantiate and construct DirectByteBuffer
URL: https://github.com/apache/flink/pull/11160
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
ASF GitHub Bot
logged work - 20/Feb/20 14:11
-
- Time Spent:
- 10m
-
azagrebin commented on pull request #11160: [
FLINK-15094] Use Unsafe to instantiate and construct DirectByteBuffer
URL: https://github.com/apache/flink/pull/11160
## What is the purpose of the change
If we use reflection to create a DirectByteBuffer to wrap unsafe native memory allocations, it causes illegal access warnings in Java9+.
This PR changes this to use Unsafe to instantiate a DirectByteBuffer. The address and capacity fields are set by direct unsafe memory operations. Other fields are set by calling ByteBuffer#clear at the end.
Unsafe operations skips the illegal access verification and do not result in warnings. The same approach is used to get the address field.
This solution still relies on Unsafe which is about to be removed in future Java releases. If it is removed and we still do not want to contribute to direct memory by allocating native managed memory, we will have to find an alternative solution, like e.g. writing a custom native allocator and use JNI API to instantiate the wrapping DirectByteBuffer (NewDirectByteBuffer in C).
## Verifying this change
unit tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org