-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Provide new
FFmpegFrameGrabber(InputStream)
and `FFmpegFrameReco…
…rder(OutputStream)` constructors (issue #95) * Make `FrameFilter`, `FrameGrabber`, and `FrameRecorder` implement `Closeable` to let us try-with-resources
- Loading branch information
Showing
7 changed files
with
271 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
102 changes: 102 additions & 0 deletions
102
platform/src/test/java/org/bytedeco/javacv/FrameGrabberTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
/* | ||
* Copyright (C) 2016 Samuel Audet | ||
* | ||
* Licensed either under the Apache License, Version 2.0, or (at your option) | ||
* under the terms of the GNU General Public License as published by | ||
* the Free Software Foundation (subject to the "Classpath" exception), | ||
* either version 2, or any later version (collectively, the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* http://www.gnu.org/licenses/ | ||
* http://www.gnu.org/software/classpath/license.html | ||
* | ||
* or as provided in the LICENSE.txt file that accompanied this code. | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package org.bytedeco.javacv; | ||
|
||
import java.io.File; | ||
import java.io.FileInputStream; | ||
import java.io.FileOutputStream; | ||
import org.bytedeco.javacpp.Loader; | ||
import org.bytedeco.javacpp.indexer.UByteIndexer; | ||
import org.junit.Test; | ||
|
||
import static org.bytedeco.javacpp.avutil.*; | ||
import static org.junit.Assert.*; | ||
|
||
/** | ||
* Test cases for FrameGrabber classes. Also uses other classes from JavaCV. | ||
* | ||
* @author Samuel Audet | ||
*/ | ||
public class FrameGrabberTest { | ||
|
||
@Test public void testFFmpegFrameGrabber() { | ||
System.out.println("FFmpegFrameGrabber"); | ||
|
||
File tempFile = new File(Loader.getTempDir(), "test.mkv"); | ||
try { | ||
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(new FileOutputStream(tempFile), 640, 480); | ||
recorder.setFormat("matroska"); // mp4 doesn't support streaming | ||
recorder.setPixelFormat(AV_PIX_FMT_BGR24); | ||
recorder.setVideoCodecName("libx264rgb"); | ||
recorder.setVideoQuality(0); // lossless | ||
recorder.start(); | ||
|
||
Frame[] frames = new Frame[1000]; | ||
for (int n = 0; n < frames.length; n++) { | ||
Frame frame = new Frame(640, 480, Frame.DEPTH_UBYTE, 3); | ||
UByteIndexer frameIdx = frame.createIndexer(); | ||
for (int i = 0; i < frameIdx.rows(); i++) { | ||
for (int j = 0; j < frameIdx.cols(); j++) { | ||
for (int k = 0; k < frameIdx.channels(); k++) { | ||
frameIdx.put(i, j, k, n + i + j + k); | ||
} | ||
} | ||
} | ||
recorder.record(frame); | ||
frames[n] = frame; | ||
} | ||
recorder.stop(); | ||
recorder.release(); | ||
|
||
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(new FileInputStream(tempFile)); | ||
grabber.start(); | ||
|
||
for (int n = 0; n < frames.length; n++) { | ||
Frame frame = frames[n]; | ||
Frame frame2 = grabber.grab(); | ||
assertEquals(frame.imageWidth, frame2.imageWidth); | ||
assertEquals(frame.imageHeight, frame2.imageHeight); | ||
assertEquals(frame.imageChannels, frame2.imageChannels); | ||
|
||
UByteIndexer frameIdx = frame.createIndexer(); | ||
UByteIndexer frame2Idx = frame2.createIndexer(); | ||
for (int i = 0; i < frameIdx.rows(); i++) { | ||
for (int j = 0; j < frameIdx.cols(); j++) { | ||
for (int k = 0; k < frameIdx.channels(); k++) { | ||
int b = frameIdx.get(i, j, k); | ||
assertEquals(b, frame2Idx.get(i, j, k)); | ||
} | ||
} | ||
} | ||
} | ||
assertEquals(grabber.grab(), null); | ||
grabber.stop(); | ||
grabber.release(); | ||
} catch (Exception e) { | ||
fail("Exception should not have been thrown: " + e); | ||
} finally { | ||
tempFile.delete(); | ||
} | ||
} | ||
|
||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Saudet,
I hit the memory leak issue when I implement the video and audio stream with the current feature.
The implementation steps(take video for example) for me:
The pseudocodes are attached below:
I reproduce the issue via opening the video files.
I want to know whether I lost some codes when calling the related classes/methods or not.
Can you give me some suggestions for the issue?
Thanks in advance.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which method should I call?
recorder.release()
orgrabber.release()
.And when should I call it? I have to hold the inputstream all the time.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Saudet.
The two objects "recorder" and "grabber" need to be enabled all the time to process the streams.
So, it's hard for me to stop and freee the resource for the two objects.
Even if I don't free the resource of the two objects, it should leak the resource of the two objects occupied once only.
Why does it leak more and more?
Need I free the resource of the frame object("frame1" in my codes above) in the "while" loop?
How?
Thanks.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It maybe
I commented the codes above and find that it seems to be fine.
If the usage of the stream for javacv in my first comments is fine, I can investigate more about my codes.
Thanks for your response.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Saudet,
I export the dump file with jmap tool, and analyse the memory for the codes with MAT(Eclipse Memory Analyzer tool).
Its "Leak Hunter" shows that:
One instance of "java.io.BufferedInputStream" loaded by "" occupies 536,870,968 (87.56%) bytes. The instance is referenced by org.bytedeco.javacv.FFmpegFrameGrabber @ 0x838f53f0 , loaded by "sun.misc.Launcher$AppClassLoader @ 0x80a99560". The memory is accumulated in one instance of "byte[]" loaded by "< system class loader >".
It seems to be that the memory is not free in BufferedInputStream object(constructed with PipedInputStream object I set) in FFmpegFrameGrabber.
Can you have a look please?
Or give me some suggestions?
Thanks.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yes, that's a limitation of using InputStream with FFmpeg. We basically have to load the whole file in memory: https://github.com/bytedeco/javacv/blob/master/src/main/java/org/bytedeco/javacv/FFmpegFrameGrabber.java#L751
Some formats like Matroska can do better, so if you don't need to seek, you could try to reduce that value.
16050a2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, thanks Saudet.
Let me have a try, and investigate how to fix it better.