Audio/video codec control for lag and limited/spurious available bandwidth

Product requests and ideas
1

Hi,

Testing JAMI and having a lot of video quality&lag issues, less so with audio but still some there.
Particularly with slower processor even over fixed 100Megabit plus service at both ends.

My thought is

I would like to be able to (on the fly) configure the codec video size and rates up or down in the client (at either end for the conversation). So if I am mobile with crappy service, I can downgrade my client to (perhaps one preferred) lower resolution & frame rate. If the client is allowed to self reconfigure on the fly the having a auto/sticky/stuck (better as is now/ stay on lower resolution for xx seconds of higher bandwidth indication / stay downgraded until user takes action) configuration setting (not an on the fly setting) .

This way making calls I may be able to minimize the lag due to less data AND less data loss.

Also, I would like to note that when video interruptions occur, there is this “catch up” video where the video spools at faster rate until it reaches current frames. The catch up video is not useful for my use case, I would like to not see the catch up frames and just jump forward to current time marker… would lowering the buffer size or discarding frames more than .xx seconds late accomplish this?

I think there are some really good ideas on this forum maybe there is a need to be grouped together where similar for implementation?

With kind regards,
Rob

1 Like
2

The following instructions are for Jami for Desktop (GNU/Linux, macOS, and Windows); in Jami:

  1. Click the “Open settings” cogwheel.
  2. Click the “Media” tab.
  3. Click the “Video” heading.
  4. Change both the “Resolution” and “Frames per second” to lower values.

If you are sharing your screen or window, change the setting after clicking the “Screen sharing” heading.

Please ask the person you are communicating with on Jami to do the same if the problem still persists.

Thank you

3

Hi, thanks for answering,

I tested this and I found that I have to end and restart the call in order for settings to take effect.

The point of my comment was to do this dynamically as needed.

With kind regards,
Rob

1 Like
4

Hi,
I have to agree with Rob on this. It would be helpful, if the user settings are taken as a maximum and Jami adjusts the resolution and/ or frame rate dynamically depending on hardware, bandwidth,etc. (similar to the “competition”).

Why? I have been using Skype on Linux for the last couple of years to stay in touch with my elderly parents (I life on a different continent). They know how to use a browser and Skype but not much else as far as computers are concerned. After the EoL of Skype I am looking for alternatives. I installed Jami remotely and gave instructions on how to use it. It mostly works fine, but the their video deteriorates every 30 seconds or so. Troubleshooting with a non-tech user on the other side is difficult. It would be great, if the system can sort itself out.

Regards,
Matthias

5

Hi, maybe it is possible to work this issue through a plugin/extension?

I am thinking if the Jami client has the hooks available to make this an extension that might be perfect as we could then have an overlay or steal a portion of the UI?

As I understand it we have MediaHandler and ChatHandler accessible through the current Plugin/Extension schema. Where are the definitions for these with the allowed control/calls?
The documentation to create a plugin/extension only gives a hello world example that I can find in my limited search…

With kind regards,
Rob

6

Plugins can replace outgoing frames (not encoded) or incoming frames (decoded), but cannot handle the encoding process and do bandwidth estimation.

7

Thank you @Rubus_chiliadenus.

With the current settings, we can change the client settings in a call, but the client does not change the frame size and rate during the call. Would this be considered an issue?

I am wondering if transcoding the frames in an extension (I know it would be processor intensive) as a man in the middle between the camera and the lrc encoder is possible. Then we could leave the gorilla (jami client) to be in peace? I still have not located the Media and Chat handler apis, I need to look further for these.

If we cannot through extension then client or lrc has to be modified to allow codec and rate control.

8

I think changing rate manually during the call should not be allowed if auto-adjustment of the bitrate is enabled.

See