WebRTC的peerconnection客户端如何实现长尾词的实时互动问答？

本文共计1519个文字，预计阅读时间需要7分钟。

准备+简单+带麦克风和摄像头的电脑+复杂+不带麦克风和摄像头的电脑+需要安装OBS软件（虚拟摄像头）+代码方面+禁用音频+void Conductor::AddTracks() { if (!peer_connection_->GetSender()) {

准备

简单

两台带有麦克风和摄像头的电脑

复杂

没有麦克风和摄像头的单台电脑需要安装OBS软件（虚拟摄像头）

代码方面需要禁用音频

void Conductor::AddTracks() { if (!peer_connection_->GetSenders().empty()) { return; // Already added tracks. } // rtc::scoped_refptr<webrtc::AudioTrackInterface> audio_track( // peer_connection_factory_->CreateAudioTrack( // kAudioLabel, // peer_connection_factory_->CreateAudioSource(cricket::AudioOptions()) // .get())); // auto result_or_error = peer_connection_->AddTrack(audio_track, {kStreamId}); // if (!result_or_error.ok()) { // RTC_LOG(LS_ERROR) << "Failed to add audio track to PeerConnection: " // << result_or_error.error().message(); // } rtc::scoped_refptr<CapturerTrackSource> video_device = CapturerTrackSource::Create(); if (video_device) { rtc::scoped_refptr<webrtc::VideoTrackInterface> video_track_( peer_connection_factory_->CreateVideoTrack(video_device, kVideoLabel)); main_wnd_->StartLocalRenderer(video_track_.get()); auto result_or_error = peer_connection_->AddTrack(video_track_, {kStreamId}); if (!result_or_error.ok()) { RTC_LOG(LS_ERROR) << "Failed to add video track to PeerConnection: " << result_or_error.error().message(); } } else { RTC_LOG(LS_ERROR) << "OpenVideoCaptureDevice failed"; } main_wnd_->SwitchToStreamingUI(); }

OBS版本：OBS-Studio-29.1.3-Full-Installer-x64.exe

配置参考

手把手！教你用新版OBS虚拟摄像头 - 哔哩哔哩 (bilibili.com)

可以使用自动配置向导，选择最后一项“我只使用虚拟摄像机”即可

回到主界面，在下方的“来源”栏下点击“+”加号，添加一个“媒体源”

点击“浏览”，选择刚才录制好的视频文件

选择好文件后，一定记得勾选“循环”！

单击主界面右下角“控件”一栏里的“启动虚拟摄像机”

WebRTC服务器配置

src\examples\peerconnection\client\flag_defs.h

ABSL_FLAG(std::string, server, "localhost", "The server to connect to.");

定义了连接WebRTC服务器IP地址

修改为指定的服务器IP地址

src\examples\peerconnection\client\defaults.cc

const uint16_t kDefaultServerPort = 8888;

定义了连接WebRTC服务器端口

操作

任意一方可以在中途按ESC键退出音视频通信

运行失败按照如下修改代码

依次运行exe文件peerconnection_server和peerconnection_client，可以发现server窗口中出现了新的连接字样，但是client并没有跳转画面，依旧卡在初始界面

方法一

运行PeerConnectionClient，使用本地ip连接PeerConnectionServer时，PeerConnectionClient的onConnect没有回调。经过分析，是由于main.cc定义的线程没有运行

需要在peer_connection_client.cc中的方法PeerConnectionClient::DoConnect()添加逻辑，在方法最后添加rtc::Thread::Current()->Start();

void PeerConnectionClient::DoConnect() { control_socket_.reset(CreateClientSocket(server_address_.ipaddr().family())); hanging_get_.reset(CreateClientSocket(server_address_.ipaddr().family())); InitSocketSignals(); char buffer[1024]; snprintf(buffer, sizeof(buffer), "GET /sign_in?%s HTTP/1.0\r\n\r\n", client_name_.c_str()); onconnect_data_ = buffer; bool ret = ConnectControlSocket(); if (ret) state_ = SIGNING_IN; if (!ret) { callback_->OnServerConnectionFailure(); } rtc::Thread::Current()->Start(); }

另外还有个线程死等的问题，要修改peer_connection_client.cc中的Wait方法

windows运行webrtc的PeerConnectionClient连接问题修复_peerconnectionclient 空白_vsix_ling的博客-CSDN博客

方法二

/* * Copyright 2012 The WebRTC Project Authors. All rights reserved. * * Use of this source code is governed by a BSD-style license * that can be found in the LICENSE file in the root of the source * tree. An additional intellectual property rights grant can be found * in the file PATENTS. All contributing project authors may * be found in the AUTHORS file in the root of the source tree. */ // clang-format off // clang formating would change include order. #include <windows.h> #include <shellapi.h> // must come after windows.h // clang-format on #include <string> #include <vector> #include "absl/flags/parse.h" #include "examples/peerconnection/client/conductor.h" #include "examples/peerconnection/client/flag_defs.h" #include "examples/peerconnection/client/main_wnd.h" #include "examples/peerconnection/client/peer_connection_client.h" #include "rtc_base/checks.h" #include "rtc_base/ssl_adapter.h" #include "rtc_base/string_utils.h" // For ToUtf8 #include "rtc_base/win32_socket_init.h" #include "system_wrappers/include/field_trial.h" #include "test/field_trial.h" #include "rtc_base/logging.h" #include "rtc_base/log_sinks.h" namespace { // A helper class to translate Windows command line arguments into UTF8, // which then allows us to just pass them to the flags system. // This encapsulates all the work of getting the command line and translating // it to an array of 8-bit strings; all you have to do is create one of these, // and then call argc() and argv(). class WindowsCommandLineArguments { public: WindowsCommandLineArguments(); WindowsCommandLineArguments(const WindowsCommandLineArguments&) = delete; WindowsCommandLineArguments& operator=(WindowsCommandLineArguments&) = delete; int argc() { return argv_.size(); } char** argv() { return argv_.data(); } private: // Owned argument strings. std::vector<std::string> args_; // Pointers, to get layout compatible with char** argv. std::vector<char*> argv_; }; WindowsCommandLineArguments::WindowsCommandLineArguments() { // start by getting the command line. LPCWSTR command_line = ::GetCommandLineW(); // now, convert it to a list of wide char strings. int argc; LPWSTR* wide_argv = ::CommandLineToArgvW(command_line, &argc); // iterate over the returned wide strings; for (int i = 0; i < argc; ++i) { args_.push_back(rtc::ToUtf8(wide_argv[i], wcslen(wide_argv[i]))); // make sure the argv array points to the string data. argv_.push_back(const_cast<char*>(args_.back().c_str())); } LocalFree(wide_argv); } } // namespace class CustomSocketServer : public rtc::PhysicalSocketServer { public: bool Wait(webrtc::TimeDelta max_wait_duration, bool process_io) override { if (!process_io) return true; return rtc::PhysicalSocketServer::Wait(webrtc::TimeDelta::Zero(), process_io); } }; int PASCAL wWinMain(HINSTANCE instance, HINSTANCE prev_instance, wchar_t* cmd_line, int cmd_show) { rtc::WinsockInitializer winsock_init; //rtc::PhysicalSocketServer ss; CustomSocketServer ss; rtc::AutoSocketServerThread main_thread(&ss); rtc::FileRotatingLogSink* pLog = new rtc::FileRotatingLogSink("D:/", "webrtc.log", 1024*1024*10, 10); pLog->Init(); rtc::LogMessage::AddLogToStream(pLog, rtc::LS_VERBOSE); WindowsCommandLineArguments win_args; int argc = win_args.argc(); char** argv = win_args.argv(); absl::ParseCommandLine(argc, argv); // InitFieldTrialsFromString stores the char*, so the char array must outlive // the application. const std::string forced_field_trials = absl::GetFlag(FLAGS_force_fieldtrials); webrtc::field_trial::InitFieldTrialsFromString(forced_field_trials.c_str()); // Abort if the user specifies a port that is outside the allowed // range [1, 65535]. if ((absl::GetFlag(FLAGS_port) < 1) || (absl::GetFlag(FLAGS_port) > 65535)) { printf("Error: %i is not a valid port.\n", absl::GetFlag(FLAGS_port)); return -1; } const std::string server = absl::GetFlag(FLAGS_server); MainWnd wnd(server.c_str(), absl::GetFlag(FLAGS_port), absl::GetFlag(FLAGS_autoconnect), absl::GetFlag(FLAGS_autocall)); if (!wnd.Create()) { RTC_DCHECK_NOTREACHED(); return -1; } rtc::InitializeSSL(); PeerConnectionClient client; auto conductor = rtc::make_ref_counted<Conductor>(&client, &wnd); main_thread.Start(); // Main loop. MSG msg; BOOL gm; while ((gm = ::GetMessage(&msg, NULL, 0, 0)) != 0 && gm != -1) { if (!wnd.PreTranslateMessage(&msg)) { ::TranslateMessage(&msg); ::DispatchMessage(&msg); } } if (conductor->connection_active() || client.is_connected()) { while ((conductor->connection_active() || client.is_connected()) && (gm = ::GetMessage(&msg, NULL, 0, 0)) != 0 && gm != -1) { if (!wnd.PreTranslateMessage(&msg)) { ::TranslateMessage(&msg); ::DispatchMessage(&msg); } } } rtc::CleanupSSL(); return 0; }

注意事项

默认启动后是 localhost ，不知道为什么我必须改成 127.0.0.1 才可以连接服务器成功。 翻阅资料得出结果是因为 windows 把 localhost 解析为 ipv6 地址 ::1 而不是 127.0.0.1。查了hosts 把ipv6已经屏蔽了，127.0.0.1 localhost 也有，就是不能正常解析

退出崩溃解决

去掉RTC_NORETURN提示错误

function declared 'noreturn' should not return [-Werror,-Winvalid-noreturn]

修改代码如下

RTC_NORETURN void WriteFatalLog(absl::string_view output) { #if defined(WEBRTC_ANDROID) std::string output_str(output); __android_log_print(ANDROID_LOG_ERROR, RTC_LOG_TAG_ANDROID, "%s\n", output_str.c_str()); #endif fflush(stdout); fwrite(output.data(), output.size(), 1, stderr); fflush(stderr); #if defined(WEBRTC_WIN) //DebugBreak(); #endif exit(0); //abort(); }

外网扩展

Windows下 WebRTC Demo运行： PeerConnection_window.rtcpeerconnection文档_chenxiemin的博客-CSDN博客

参考

fix.patch (1.2 KB) - webrtc - Web-based real-time communication - Monorail (chromium.org)

13498 - the peerconnection_client on windows not displaying the video on win10 - webrtc (chromium.org)

WebRTC源码demo无法运行_陈成不姓丞的博客-CSDN博客