FFmpegFrameRecorder didn't work with OutputStream in Android

[Like-fire]

In my Android test application FFmpegFrameRecorder work properly when i pass File in conctructor, but when i pass OutputStream, app crashes and in Logcat i can see only com.github.crazyorr.ffmpegrecorder A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 29234 (Thread-266)
Code, where i init my frame recorder:
String recordedTime = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
File file = CameraHelper.getOutputMediaFile(recordedTime, CameraHelper.MEDIA_TYPE_VIDEO);
FileOutputStream fos = new FileOutputStream(file);
mFrameRecorder = new FFmpegFrameRecorder(fos, videoWidth, videoHeight, 1);
mFrameRecorder.setFormat("mp4");
mFrameRecorder.setSampleRate(sampleAudioRateInHz);
mFrameRecorder.setFrameRate(frameRate);
mFrameRecorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
mFrameRecorder.setVideoOption("crf", "28");
mFrameRecorder.setVideoOption("preset", "superfast");
mFrameRecorder.setVideoOption("tune", "zerolatency");
Gradle:

compile(group: 'org.bytedeco', name: 'javacv-platform', version: '1.3.2') {
        exclude group: 'org.bytedeco.javacpp-presets'
    }
    compile group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.2.1-1.3'
    compile group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.2.1-1.3', classifier: 'android-arm'

Before that i use FFmpegFrameRecorder via just passing file in constructor, but in my app i need possibility to write video from camera using OutputStream.
Please help, maybe i understand something wrong

[saudet]

Does the same thing happen with Java SE on another platform, or does it happen only with Android?

[Like-fire]

I tried do that only on Android platform, on devices: Nexus 5X, Samsung SGS 6, Xiaomi Mi5s.
But didn't try it anywhere else, because it doesn't needed for me.
Maybe you can give small sample, where FFmpefFrameRecorder works fine with OutlutStream in Android?

[saudet]

@peter9870 Were you able to use OutputStream under Android?

[Like-fire]

I want to create the app, which will help in recording video from android camera to cloud(like dropbox) file directly, and i hope that your JavaCV lib with OutputStream param will helps me in it

[saudet]

I hope so too, but someone has to spend time to figure it out :)

[Like-fire]

Maybe you have any example how to use FFMpegFrameRecorder with OutputStream? Or this crash platform-specific and refers to Android OS?

[pgfsim]

@saudet Yes I got it working. @Like-fire You should use recorder.setFormat("matroska"); You can still keep your output as a .mp4 file though.

I needed to be able to save to SD so I had to use ParcelFileDescriptor like this...

DocumentFile gjDir = vidsavelocationSD.createFile("video/mkv",videotimeStamp+".mp4");

                OutputStream out = null;

                try {
                    ParcelFileDescriptor pfd =
                            getContentResolver().
                                    openFileDescriptor(gjDir.getUri(), "rw");

                    out = getContentResolver().openOutputStream(gjDir.getUri());

                } catch (FileNotFoundException e) {
                    e.printStackTrace();
                }

I'm not sure if this applies to you but I also had to set the pixelformat like so otherwise it wouldn't work correctly.
recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);

[Like-fire]

Yes, it works! Thank you, but i have quetions. Using this camera i had bad video quality. When i change crv and set value "6" from example camera doesn't respond to fast movements, when i set value "35" camera respond much faster but, too much noise. How i can improve video quality. Thank you very much!

[pgfsim]

Have you tried using recorder.setVideoQuality(10);

[Like-fire]

It solved issue with bad picture quality, but absolutely not solve problem with video freezes on fast camera moving. It looks like too few frames per second, but i set it to 30

[saudet]

What do you mean by "freeze"? The application doesn't recover?

[Like-fire]

No, just looks like there are about 10 frames per second, but it set to 30

[saudet]

So, your device is just too slow to encode as fast as you would like, that's normal.

[Like-fire]

On recording CPU monitor show less then 30-40%, and i tried it on phones: Google Pixel XL, Xiaomi Mi5S, LG Nexus 5X, Samsung S7. All of them show similar results. I think the problem has other reasons.

[saudet]

So, the codec you are using doesn't support multithreading. Try to use a codec that supports multithreading, and use more than one thread for encoding.

[anonym24]

@peter9870 does your recorded videos this way have problem with determine video length in video player apps? like this #95 (comment)

[anonym24]

so works with mastorka, but problem with seeking and length detection, seems when using outputStream ffmpeg mb cannot add some final lines to the video file when we stop recording

[pgfsim]

@anonym24 yes I've seen this issue where the video has length 00:00

[anonym24]

@peter9870 it's always undefined
#839 - was said it has some limitations

[cfcodefans]

Does the same thing happen with Java SE on another platform, or does it happen only with Android?

yes, I encounter this problem on win7 java10 environment

@Test
    fun testFFmpegFrameRecorder() {
        val inputFilename = "./test-videos/wildlife.wmv"
        val outputFilename = "./test-videos/encoded.mp4"

        val outputPath = Paths.get(outputFilename)
        Files.deleteIfExists(outputPath)
        Files.createFile(outputPath)

        FileInputStream(File(inputFilename)).use {
            FFmpegFrameGrabber(it).use { fg ->
                fg.imageMode = FrameGrabber.ImageMode.RAW
                fg.start()
                val frameSeq = generateSequence { fg.grab().takeIf { frame -> frame != null } }
                val frameNum = fg.lengthInFrames

                val fr = FFmpegFrameRecorder(
                        //outputFilename,//
                        FileOutputStream(outputFilename),
                        640,//must be multiplication of 32
                        320,//must be even
                        fg.audioChannels)//.use { fr ->

                fr.format = "mp4"
                fr.isInterleaved = true
                fr.pixelFormat = AV_PIX_FMT_YUV420P
                fr.frameRate = fg.frameRate
                fr.videoBitrate = fg.videoBitrate
                fr.audioBitrate = fg.audioBitrate
                fr.videoCodec = AV_CODEC_ID_H264

                fr.start()
                frameSeq.forEachIndexed { i: Int, fra: Frame ->
                    if (i % 100 == 0) {
                        log.info("$i/$frameNum\t${Jsons.toString(mapOf("timestamp" to fra.timestamp))}")
                    }
                    fr.record(fra)
                }
                fr.stop()
                fr.close()
            }
        }
    }

it failed at

JRE version: OpenJDK Runtime Environment (10.0+46) (build 10+46)
 Java VM: OpenJDK 64-Bit Server VM (10+46, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
 Problematic frame:
 C  [avformat-58.dll+0xe1abd]

j  org.bytedeco.javacpp.avformat.av_interleaved_write_frame(Lorg/bytedeco/javacpp/avformat$AVFormatContext;Lorg/bytedeco/javacpp/avcodec$AVPacket;)I+0
j  org.bytedeco.javacv.FFmpegFrameRecorder.writePacket(ILorg/bytedeco/javacpp/avcodec$AVPacket;)V+75
j  org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(IIIIII[Ljava/nio/Buffer;)Z+947
j  org.bytedeco.javacv.FFmpegFrameRecorder.record(Lorg/bytedeco/javacv/Frame;I)V+70
j  org.bytedeco.javacv.FFmpegFrameRecorder.record(Lorg/bytedeco/javacv/Frame;)V+3

any hints? thanks in advance

[saudet]

@cfcodefans Could you attach sample media files to reproduce this issue?

[cfcodefans]

yeah, actually I think I have got close to the bottom of this problem

in class org.bytedeco.javacv.FFmpegFrameRecorder#startUnsafe
line 396:

if (this.outputStream != null) {
                this.avio = avformat.avio_alloc_context(new BytePointer(avutil.av_malloc(4096L)), 4096, 1, this.oc, (Read_packet_Pointer_BytePointer_int)null, writeCallback, (Seek_Pointer_long_int)null);
                this.oc.pb(this.avio);
                this.filename = this.outputStream.toString();
                outputStreams.put(this.oc, this.outputStream);
            }

(Seek_Pointer_long_int)null is parameter which is needed in conversion for some format like mp4
I have not dug very deep, it seems that mp4 format conversion needs to write back some metadata back to head of buffer?
It seems to be difficult to work with outputstream.... what if i pass a ByteArrayOutputStream? basically I aim to process video from stream to stream without writing to any file on disk.....

[saudet]

We could add a callback for seek and make it work for specialized implementations like ByteArrayOutputStream, yes, that's a good idea! Could you give it a try?

[cfcodefans]

sure, will try

[svorlauf]

Is there any progress on implementing this feature?

[saudet]

Thanks to @svorlauf, we can now use SeekableByteArrayOutputStream for this, see pull #1350.
Please give it all a try with the snapshots before the next release: http://bytedeco.org/builds/

[saudet]

JavaCV 1.5.3 has been released with SeekableByteArrayOutputStream. Thanks again @svorlauf!