Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deviceId of AudioDeviceManager is not shareable with other package #960

Open
nguyenhy opened this issue Mar 17, 2023 · 13 comments
Open

deviceId of AudioDeviceManager is not shareable with other package #960

nguyenhy opened this issue Mar 17, 2023 · 13 comments

Comments

@nguyenhy
Copy link

nguyenhy commented Mar 17, 2023

Describe the bug
We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

We decide to use another package to make this, since we can't stream from the one cam/mic and preview the exact cam/mic to allow user previews their cam/mic. Because we can't find a solution to use the same cam/mic one for stream and one for preview, and these 2 have different configs. Even, when we create 2 different RtcEngine with the same/different appId

With record and camera_macos. We were able to make it but there's one problem.
The deviceId return from these 2 packages is different than the deviceId from Agora SDK

To Reproduce
Note that I plugin in 2 similar camera

final engine = createAgoraRtcEngine();
engine
    .initialize(
  const RtcEngineContext(
    appId: appId,
  ),
)
    .then((value) async {
  final videoManager = engine.getVideoDeviceManager();
  final audioManager = engine.getAudioDeviceManager();
  final videoDevices = await videoManager.enumerateVideoDevices();
  final audioDevices = await audioManager.enumerateRecordingDevices();

  for (var element in videoDevices) {
    print(['videoDevices', element.deviceId, element.deviceName]);
  }
  /// [Log] 
  /// flutter: [videoDevices, 0x2100000046d0825, USB Camera VID:1133 PID:2085]
  /// flutter: [videoDevices, 0x2200000046d0825, USB Camera VID:1133 PID:2085]

  for (var element in audioDevices) {
    print(['audioDevices', element.deviceId, element.deviceName]);
  }
  /// flutter: [audioDevices, 99, default (Unknown USB Audio Device)]
  /// flutter: [audioDevices, 95, Unknown USB Audio Device]
});


final record = Record();
record.listInputDevices().then((inputDevices) {
  for (var element in inputDevices) {
    print([
      'record',
      '|',
      element.id,
      '|',
      element.label,
      '|',
      element.channels,
    ]);
  }
});
/// Log
/// flutter: [record, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3, |, Unknown USB Audio Device, |, null]
/// flutter: [record, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3, |, Unknown USB Audio Device, |, null]


CameraMacOS.instance
    .listDevices(deviceType: CameraMacOSDeviceType.video)
    .then((value) {
  for (var element in value) {
    print([
      'CameraMacOSDeviceType.video',
      '|',
      element.deviceId,
      '|',
      element.localizedName,
      '|',
      element.manufacturer,
    ]);
  }
});
/// Log
/// flutter: [CameraMacOSDeviceType.video, |, 0x2100000046d0825, |, USB Camera VID:1133 PID:2085, |, Unknown]
/// flutter: [CameraMacOSDeviceType.video, |, 0x2200000046d0825, |, USB Camera VID:1133 PID:2085, |, Unknown] 

CameraMacOS.instance
    .listDevices(deviceType: CameraMacOSDeviceType.audio)
    .then((value) {
  for (var element in value) {
    print([
      'CameraMacOSDeviceType.audio',
      '|',
      element.deviceId,
      '|',
      element.localizedName,
      '|',
      element.manufacturer,
    ]);
  }
});
/// Log
/// flutter: [CameraMacOSDeviceType.audio, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3, |, Unknown USB Audio Device, |, Unknown Manufacturer]
/// flutter: [CameraMacOSDeviceType.audio, |, AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3, |, Unknown USB Audio Device, |, Unknown Manufacturer]

As shown in the above sample code, record, and camera_macos are able to return the same id for audio input devices, whichs is AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:D5ACA760:3 and AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3.
But the id from enumerateRecordingDevices return is 95, 99.

$ system_profiler SPUSBDataType
USB:

    USB 3.0 Bus:

      Host Controller Driver: AppleEmbeddedUSBXHCIFL1100
      PCI Device ID: 0x1100
      PCI Revision ID: 0x0010
      PCI Vendor ID: 0x1b73

        Miscellaneous Device:

          Product ID: 0x0825
          Vendor ID: 0x046d  (Logitech Inc.)
          Version: 0.12
          Serial Number: 33829440
          Speed: Up to 480 Mb/s
          Location ID: 0x02200000 / 2
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

        Miscellaneous Device:

          Product ID: 0x0825
          Vendor ID: 0x046d  (Logitech Inc.)
          Version: 0.12
          Serial Number: D5ACA760
          Speed: Up to 480 Mb/s
          Location ID: 0x02100000 / 1
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

Expected behavior
method enumerateRecordingDevices return value ofdeviceId as other package

Desktop (please complete the following information):

  • macOS: 13.0.1
  • SDK: 6.1.0
@littleGnAl
Copy link
Collaborator

We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

I'm so sorry that I might miss your point, is that mean you need to preview all the video/audio devices at the same time?

@littleGnAl littleGnAl added the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023
@nguyenhy
Copy link
Author

We're making a desktop app for Windows and macOS users. In this app, there's a feature that allows the user to preview their camera and microphone in case users have more than one video/audio input device.

I'm so sorry that I might miss your point, is that mean you need to preview all the video/audio devices at the same time?

So we need to preview 1 video/audio input device at a time while using it to push to channel. Like, preview a cam/mic before use it to push stream.

@github-actions github-actions bot removed the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023
@littleGnAl
Copy link
Collaborator

In our example, you can change and preview the device, and then you can control when to push with joinChannel. I'm not very sure if it meets your requirement.
https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/main/example/lib/examples/advanced/device_manager/device_manager.dart

@littleGnAl littleGnAl added the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023
@nguyenhy
Copy link
Author

And the main reason for this issue is that we need to show the volume meter of current preview mic. I found that record package allow use to do this.
But deviceId from record and agora_rtc_engine is not the same. So even if we finish the feature, we can't select device using AudioDeviceManager.setRecordingDevice(deviceId)

@nguyenhy
Copy link
Author

In our example, you can change and preview the device, and then you can control when to push with joinChannel. I'm not very sure if it meets your requirement. https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/main/example/lib/examples/advanced/device_manager/device_manager.dart

So our problem is that, saying we have 2 cameras: camA and camB. UserA is using camA to push the stream to other users in the channel and the user wants to swap the camera to camB. But he wants to preview it before making the change.

So our feature is that we will show a list of the available cameras inside a preview dialog. User can choose and preview it. Once he feels good he can click on the OK button to use this camera to push stream to other users in the channel.

Somehow it looks like this
Screenshot 2023-03-17 at 15 49 52

So, at the time they preview, we have 2 AgoraVideoView. One in the preview dialog and one in the main view.
If we use VideoDeviceManager.setDevice(deviceId). Then it also means the main player is also changed to current preview camera.

@github-actions github-actions bot removed the waiting for customer response waiting for customer response, or closed by no-reponse bot label Mar 17, 2023
@littleGnAl
Copy link
Collaborator

Thanks for your detail, at this time our SDK only supports 2 cameras preview, you can try the VideoSourceType.videoSourceCameraSecondary, I still need more info on how to preview others' video devices internally.

@nguyenhy
Copy link
Author

I still need more info on how to preview others' video devices internally.

I would love to know how to do it.
Thanks for your support

@qiuyanli1990
Copy link

@nguyenhy The deviceID returned by our SDK is also the deviceID returned by macOS system. Can you please try directly put the device ID returned by our SDK to the deviceID of other SDKs to see if it will work? Maybe they have different numbers but can both be used? Please let us know your test result.

@nguyenhy
Copy link
Author

nguyenhy commented Mar 27, 2023

Hi @qiuyanli1990.

Can you please try directly put the device ID returned by our SDK to the deviceID of other SDKs to see if it will work? Maybe they have different numbers but can both be used? Please let us know your test result.

I did try to use agora ID in record package but it not work.

@nguyenhy The deviceID returned by our SDK is also the deviceID returned by macOS system.

I'm really not sure about this. I try to create sample code from swift and objective-c but seem like it all return the same result. And every time I plug in and out my camera, Agora SDK returns a new different id.

    // macOS Objective-C
    NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
    for (AVCaptureDevice *device in devices) {
        NSLog(@"=======");
        NSLog(@"localizedName: %@", device.localizedName);
        NSLog(@"uniqueID: %@", device.uniqueID);
        NSLog(@"modelID: %@", device.modelID);
    
    }
=======
localizedName: Unknown USB Audio Device
uniqueID: AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:5F6D2E80:3
modelID: Unknown USB Audio Device:046D:0825
=======
localizedName: Unknown USB Audio Device
uniqueID: AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3
modelID: Unknown USB Audio Device:046D:0825
// macOS Swift
let discoverySession = AVCaptureDevice.DiscoverySession(
    deviceTypes: [.builtInMicrophone],
      mediaType: .audio, position: .unspecified
    )


for device in discoverySession.devices {
    print("===========")
    print("localizedName: ", device.localizedName);
    print("uniqueID: ", device.uniqueID);
    print("modelID: ", device.modelID);
}
===========
localizedName:  Unknown USB Audio Device
uniqueID:  AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:5F6D2E80:3
modelID:  Unknown USB Audio Device:046D:0825
===========
localizedName:  Unknown USB Audio Device
uniqueID:  AppleUSBAudioEngine:Unknown Manufacturer:Unknown USB Audio Device:33829440:3
modelID:  Unknown USB Audio Device:046D:0825

Can you help me point out how to create this int id so we can somehow create ourself adapter to retrieve String id as in uniqueID

@littleGnAl
Copy link
Collaborator

For your feature, I think you can try using startSecondaryCameraCapture to preview the selected deviceId

@nguyenhy
Copy link
Author

nguyenhy commented Mar 30, 2023

For your feature, I think you can try using startSecondaryCameraCapture to preview the selected deviceId

As I mention ealier, the main reason for this issue is that the deviceId from Agora SDK is not the device id of plugin devices.

You can read more about this in this comment

And every time I plug in and out my camera, Agora SDK returns a new different id.

@littleGnAl
Copy link
Collaborator

#960 (comment)

So we need to preview 1 video/audio input device at a time while using it to push to channel. Like, preview a cam/mic before use it to push stream.

I think this can be achieved by using the startSecondaryCameraCapture and preview it without using the other packages, you can check our example for reference.

Capture by a deviceId:

Then preview it:

@nguyenhy
Copy link
Author

Ok, I will check startSecondaryCameraCapture.

But how about deviceId of audioDevices and videoDevices not the same. And every time I plug in and out my camera, Agora SDK returns a new different id for audioDevices.

final engine = createAgoraRtcEngine();
engine
    .initialize(
  const RtcEngineContext(
    appId: appId,
  ),
)
    .then((value) async {
  final videoManager = engine.getVideoDeviceManager();
  final audioManager = engine.getAudioDeviceManager();
  final videoDevices = await videoManager.enumerateVideoDevices();
  final audioDevices = await audioManager.enumerateRecordingDevices();

  for (var element in videoDevices) {
    print(['videoDevices', element.deviceId, element.deviceName]);
  }
  /// [Log] 
  /// flutter: [videoDevices, 0x2100000046d0825, USB Camera VID:1133 PID:2085]
  /// flutter: [videoDevices, 0x2200000046d0825, USB Camera VID:1133 PID:2085]

  for (var element in audioDevices) {
    print(['audioDevices', element.deviceId, element.deviceName]);
  }
  /// flutter: [audioDevices, 99, default (Unknown USB Audio Device)]
  /// flutter: [audioDevices, 95, Unknown USB Audio Device]
});

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants