`deviceId` of `AudioDeviceManager` is not shareable with other package #960

nguyenhy · 2023-03-17T04:19:21Z

Describe the bug
We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

We decide to use another package to make this, since we can't stream from the one cam/mic and preview the exact cam/mic to allow user previews their cam/mic. Because we can't find a solution to use the same cam/mic one for stream and one for preview, and these 2 have different configs. Even, when we create 2 different RtcEngine with the same/different appId

With record and camera_macos. We were able to make it but there's one problem.
The deviceId return from these 2 packages is different than the deviceId from Agora SDK

To Reproduce
Note that I plugin in 2 similar camera

final engine = createAgoraRtcEngine();
engine
    .initialize(
  const RtcEngineContext(
    appId: appId,
  ),
)
    .then((value) async {
  final videoManager = engine.getVideoDeviceManager();
  final audioManager = engine.getAudioDeviceManager();
  final videoDevices = await videoManager.enumerateVideoDevices();
  final audioDevices = await audioManager.enumerateRecordingDevices();

  for (var element in videoDevices) {
    print(['videoDevices', element.deviceId, element.deviceName]);
  }
  /// [Log] 
  /// flutter: [videoDevices, 0x2100000046d0825, USB Camera VID:1133 PID:2085]
  /// flutter: [videoDevices, 0x2200000046d0825, USB Camera VID:1133 PID:2085]

  for (var element in audioDevices) {
    print(['audioDevices', element.deviceId, element.deviceName]);
  }
  /// flutter: [audioDevices, 99, default (Unknown USB Audio Device)]
  /// flutter: [audioDevices, 95, Unknown USB Audio Device]
});


final record = Record();
record.listInputDevices().then((inputDevices) {
  for (var element in inputDevices) {
    print([
      'record',
      '|',
      element.id,
      '|',
      element.label,
      '|',
      element.channels,
    ]);
  }
});
/// Log
/// flutter: [record, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3, |, Unknown USB Audio Device, |, null]
/// flutter: [record, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3, |, Unknown USB Audio Device, |, null]


CameraMacOS.instance
    .listDevices(deviceType: CameraMacOSDeviceType.video)
    .then((value) {
  for (var element in value) {
    print([
      'CameraMacOSDeviceType.video',
      '|',
      element.deviceId,
      '|',
      element.localizedName,
      '|',
      element.manufacturer,
    ]);
  }
});
/// Log
/// flutter: [CameraMacOSDeviceType.video, |, 0x2100000046d0825, |, USB Camera VID:1133 PID:2085, |, Unknown]
/// flutter: [CameraMacOSDeviceType.video, |, 0x2200000046d0825, |, USB Camera VID:1133 PID:2085, |, Unknown] 

CameraMacOS.instance
    .listDevices(deviceType: CameraMacOSDeviceType.audio)
    .then((value) {
  for (var element in value) {
    print([
      'CameraMacOSDeviceType.audio',
      '|',
      element.deviceId,
      '|',
      element.localizedName,
      '|',
      element.manufacturer,
    ]);
  }
});
/// Log
/// flutter: [CameraMacOSDeviceType.audio, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3, |, Unknown USB Audio Device, |, Unknown Manufacturer]
/// flutter: [CameraMacOSDeviceType.audio, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3, |, Unknown USB Audio Device, |, Unknown Manufacturer]

As shown in the above sample code, record, and camera_macos are able to return the same id for audio input devices, whichs is AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3 and AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3.
But the id from enumerateRecordingDevices return is 95, 99.

$ system_profiler SPUSBDataType
USB:

    USB 3.0 Bus:

      Host Controller Driver: AppleEmbeddedUSBXHCIFL1100
      PCI Device ID: 0x1100
      PCI Revision ID: 0x0010
      PCI Vendor ID: 0x1b73

        Miscellaneous Device:

          Product ID: 0x0825
          Vendor ID: 0x046d  (Logitech Inc.)
          Version: 0.12
          Serial Number: 33829440
          Speed: Up to 480 Mb/s
          Location ID: 0x02200000 / 2
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

        Miscellaneous Device:

          Product ID: 0x0825
          Vendor ID: 0x046d  (Logitech Inc.)
          Version: 0.12
          Serial Number: D5ACA760
          Speed: Up to 480 Mb/s
          Location ID: 0x02100000 / 1
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

Expected behavior
method enumerateRecordingDevices return value ofdeviceId as other package

Desktop (please complete the following information):

macOS: 13.0.1
SDK: 6.1.0

The text was updated successfully, but these errors were encountered:

littleGnAl · 2023-03-17T07:10:28Z

We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

I'm so sorry that I might miss your point, is that mean you need to preview all the video/audio devices at the same time?

nguyenhy · 2023-03-17T08:54:13Z

We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

I'm so sorry that I might miss your point, is that mean you need to preview all the video/audio devices at the same time?

So we need to preview 1 video/audio input device at a time while using it to push to channel. Like, preview a cam/mic before use it to push stream.

littleGnAl · 2023-03-17T09:10:53Z

In our example, you can change and preview the device, and then you can control when to push with joinChannel. I'm not very sure if it meets your requirement.
https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/main/example/lib/examples/advanced/device_manager/device_manager.dart

nguyenhy · 2023-03-17T09:43:23Z

And the main reason for this issue is that we need to show the volume meter of current preview mic. I found that record package allow use to do this.
But deviceId from record and agora_rtc_engine is not the same. So even if we finish the feature, we can't select device using AudioDeviceManager.setRecordingDevice(deviceId)

nguyenhy · 2023-03-17T09:43:34Z

In our example, you can change and preview the device, and then you can control when to push with joinChannel. I'm not very sure if it meets your requirement. https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/main/example/lib/examples/advanced/device_manager/device_manager.dart

So our problem is that, saying we have 2 cameras: camA and camB. UserA is using camA to push the stream to other users in the channel and the user wants to swap the camera to camB. But he wants to preview it before making the change.

So our feature is that we will show a list of the available cameras inside a preview dialog. User can choose and preview it. Once he feels good he can click on the OK button to use this camera to push stream to other users in the channel.

Somehow it looks like this

So, at the time they preview, we have 2 AgoraVideoView. One in the preview dialog and one in the main view.
If we use VideoDeviceManager.setDevice(deviceId). Then it also means the main player is also changed to current preview camera.

littleGnAl · 2023-03-17T09:51:11Z

Thanks for your detail, at this time our SDK only supports 2 cameras preview, you can try the VideoSourceType.videoSourceCameraSecondary, I still need more info on how to preview others' video devices internally.

nguyenhy · 2023-03-17T09:59:35Z

I still need more info on how to preview others' video devices internally.

I would love to know how to do it.
Thanks for your support

qiuyanli1990 · 2023-03-22T06:19:37Z

@nguyenhy The deviceID returned by our SDK is also the deviceID returned by macOS system. Can you please try directly put the device ID returned by our SDK to the deviceID of other SDKs to see if it will work? Maybe they have different numbers but can both be used? Please let us know your test result.

nguyenhy · 2023-03-27T03:29:45Z

Hi @qiuyanli1990.

Can you please try directly put the device ID returned by our SDK to the deviceID of other SDKs to see if it will work? Maybe they have different numbers but can both be used? Please let us know your test result.

I did try to use agora ID in record package but it not work.

@nguyenhy The deviceID returned by our SDK is also the deviceID returned by macOS system.

I'm really not sure about this. I try to create sample code from swift and objective-c but seem like it all return the same result. And every time I plug in and out my camera, Agora SDK returns a new different id.

    // macOS Objective-C
    NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
    for (AVCaptureDevice *device in devices) {
        NSLog(@"=======");
        NSLog(@"localizedName: %@", device.localizedName);
        NSLog(@"uniqueID: %@", device.uniqueID);
        NSLog(@"modelID: %@", device.modelID);
    
    }

=======
localizedName: Unknown USB Audio Device
uniqueID: AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:5F6D2E80:3
modelID: Unknown USB Audio Device:046D:0825
=======
localizedName: Unknown USB Audio Device
uniqueID: AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3
modelID: Unknown USB Audio Device:046D:0825

// macOS Swift
let discoverySession = AVCaptureDevice.DiscoverySession(
    deviceTypes: [.builtInMicrophone],
      mediaType: .audio, position: .unspecified
    )


for device in discoverySession.devices {
    print("===========")
    print("localizedName: ", device.localizedName);
    print("uniqueID: ", device.uniqueID);
    print("modelID: ", device.modelID);
}

===========
localizedName:  Unknown USB Audio Device
uniqueID:  AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:5F6D2E80:3
modelID:  Unknown USB Audio Device:046D:0825
===========
localizedName:  Unknown USB Audio Device
uniqueID:  AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3
modelID:  Unknown USB Audio Device:046D:0825

Can you help me point out how to create this int id so we can somehow create ourself adapter to retrieve String id as in uniqueID

littleGnAl · 2023-03-29T08:33:05Z

For your feature, I think you can try using startSecondaryCameraCapture to preview the selected deviceId

nguyenhy · 2023-03-30T09:24:35Z

For your feature, I think you can try using startSecondaryCameraCapture to preview the selected deviceId

As I mention ealier, the main reason for this issue is that the deviceId from Agora SDK is not the device id of plugin devices.

You can read more about this in this comment

And every time I plug in and out my camera, Agora SDK returns a new different id.

littleGnAl · 2023-03-31T09:46:37Z

#960 (comment)

So we need to preview 1 video/audio input device at a time while using it to push to channel. Like, preview a cam/mic before use it to push stream.

I think this can be achieved by using the startSecondaryCameraCapture and preview it without using the other packages, you can check our example for reference.

Capture by a deviceId:

Agora-Flutter-SDK/example/lib/examples/advanced/send_multi_camera_stream/send_multi_camera_stream.dart

Line 233 in 216eb04

deviceId: _videoDeviceInfos[1].deviceId,

Then preview it:

Agora-Flutter-SDK/example/lib/examples/advanced/send_multi_camera_stream/send_multi_camera_stream.dart

Line 174 in 216eb04

child: AgoraVideoView(

nguyenhy · 2023-03-31T10:04:20Z

Ok, I will check startSecondaryCameraCapture.

But how about deviceId of audioDevices and videoDevices not the same. And every time I plug in and out my camera, Agora SDK returns a new different id for audioDevices.

final engine = createAgoraRtcEngine();
engine
    .initialize(
  const RtcEngineContext(
    appId: appId,
  ),
)
    .then((value) async {
  final videoManager = engine.getVideoDeviceManager();
  final audioManager = engine.getAudioDeviceManager();
  final videoDevices = await videoManager.enumerateVideoDevices();
  final audioDevices = await audioManager.enumerateRecordingDevices();

  for (var element in videoDevices) {
    print(['videoDevices', element.deviceId, element.deviceName]);
  }
  /// [Log] 
  /// flutter: [videoDevices, 0x2100000046d0825, USB Camera VID:1133 PID:2085]
  /// flutter: [videoDevices, 0x2200000046d0825, USB Camera VID:1133 PID:2085]

  for (var element in audioDevices) {
    print(['audioDevices', element.deviceId, element.deviceName]);
  }
  /// flutter: [audioDevices, 99, default (Unknown USB Audio Device)]
  /// flutter: [audioDevices, 95, Unknown USB Audio Device]
});

littleGnAl added the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023

github-actions bot removed the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023

littleGnAl added the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023

github-actions bot removed the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023

littleGnAl added the keep track label Mar 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`deviceId` of `AudioDeviceManager` is not shareable with other package #960

`deviceId` of `AudioDeviceManager` is not shareable with other package #960

nguyenhy commented Mar 17, 2023 •

edited

Loading

littleGnAl commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

littleGnAl commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

littleGnAl commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

qiuyanli1990 commented Mar 22, 2023

nguyenhy commented Mar 27, 2023 •

edited

Loading

littleGnAl commented Mar 29, 2023

nguyenhy commented Mar 30, 2023 •

edited

Loading

littleGnAl commented Mar 31, 2023

nguyenhy commented Mar 31, 2023

deviceId of AudioDeviceManager is not shareable with other package #960

deviceId of AudioDeviceManager is not shareable with other package #960

Comments

nguyenhy commented Mar 17, 2023 • edited Loading

littleGnAl commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

littleGnAl commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

littleGnAl commented Mar 17, 2023

nguyenhy commented Mar 17, 2023

qiuyanli1990 commented Mar 22, 2023

nguyenhy commented Mar 27, 2023 • edited Loading

littleGnAl commented Mar 29, 2023

nguyenhy commented Mar 30, 2023 • edited Loading

littleGnAl commented Mar 31, 2023

nguyenhy commented Mar 31, 2023

`deviceId` of `AudioDeviceManager` is not shareable with other package #960

`deviceId` of `AudioDeviceManager` is not shareable with other package #960

nguyenhy commented Mar 17, 2023 •

edited

Loading

nguyenhy commented Mar 27, 2023 •

edited

Loading

nguyenhy commented Mar 30, 2023 •

edited

Loading