Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: shiguredo/sora-cpp-sdk
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 2025.4.0-canary.1
Choose a base ref
...
head repository: shiguredo/sora-cpp-sdk
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: develop
Choose a head ref

Commits on Mar 10, 2025

  1. コメントを追加する

    torikizi committed Mar 10, 2025
    Copy the full SHA
    c48153c View commit details

Commits on Apr 10, 2025

  1. Merge pull request #183 from shiguredo/feature/update-nvidia-video-co…

    …dec-sdk-13.0
    
    NVIDIA Video Codec SDK を 13.0 に上げる
    torikizi committed Apr 10, 2025
    Copy the full SHA
    92f044e View commit details

Commits on Jun 10, 2025

  1. Copy the full SHA
    7acacf7 View commit details
  2. Copy the full SHA
    8aa4f0c View commit details
  3. 修正

    voluntas committed Jun 10, 2025
    Copy the full SHA
    20c0b1a View commit details
  4. Copy the full SHA
    055502a View commit details
  5. 戻す

    voluntas committed Jun 10, 2025
    Copy the full SHA
    80151f4 View commit details
  6. Copy the full SHA
    55379d1 View commit details

Commits on Jun 12, 2025

  1. Copy the full SHA
    4631bf7 View commit details
  2. --disable-cuda を追加

    melpon committed Jun 12, 2025
    Copy the full SHA
    6318e70 View commit details
  3. CHANGES 更新

    melpon committed Jun 12, 2025
    Copy the full SHA
    4ea6603 View commit details
  4. Copy the full SHA
    3233425 View commit details

Commits on Jun 15, 2025

  1. Copy the full SHA
    f404b2d View commit details
  2. build.gradle もあげる

    torikizi committed Jun 15, 2025
    Copy the full SHA
    bc04038 View commit details
  3. Merge pull request #201 from shiguredo/feature/update-cmake-4.0.3

    CMAKE_VERSION を 4.0.3 にあげる
    torikizi authored Jun 15, 2025
    Copy the full SHA
    6b88d0b View commit details

Commits on Jun 16, 2025

  1. Copy the full SHA
    3dc97c3 View commit details
  2. ubuntu-22.04_armv8 を追加

    melpon committed Jun 16, 2025
    Copy the full SHA
    ec2660b View commit details
  3. 追加忘れ

    melpon committed Jun 16, 2025
    Copy the full SHA
    6cabec2 View commit details
  4. Copy the full SHA
    45e79f2 View commit details
  5. Copy the full SHA
    a88bb8d View commit details
  6. Copy the full SHA
    7b4eef8 View commit details
  7. Copy the full SHA
    201d809 View commit details
  8. Copy the full SHA
    3000b45 View commit details
  9. Copy the full SHA
    ea9b1da View commit details
  10. Copy the full SHA
    bcf1bbb View commit details
  11. Merge pull request #202 from shiguredo/feature/update-libwebrtc-m138

    WEBRTC_BUILD_VERSION を m138.7204.0.0 にあげる
    melpon authored Jun 16, 2025
    Copy the full SHA
    8d67325 View commit details

Commits on Jun 20, 2025

  1. Copy the full SHA
    b2d6eeb View commit details
  2. Merge pull request #203 from shiguredo/feature/add-copilot-instructions

    copilot-instructions.md を追加
    torikizi authored Jun 20, 2025
    Copy the full SHA
    7c9484c View commit details

Commits on Jun 25, 2025

  1. Copy the full SHA
    d61c0ac View commit details
  2. 記載内容を見直し

    torikizi committed Jun 25, 2025
    Copy the full SHA
    b0d38ee View commit details
  3. Merge pull request #204 from shiguredo/feature/add-known-issues

    既知の問題に AMD AMF の問題を追加
    torikizi authored Jun 25, 2025
    Copy the full SHA
    d9a0f2e View commit details
  4. Copy the full SHA
    770fd4f View commit details
  5. CHANGES を最新化

    torikizi committed Jun 25, 2025
    Copy the full SHA
    ceb2b5d View commit details
  6. Merge pull request #187 from shiguredo/feature/nvidia-video-codec-sdk…

    …-13.0-reapply
    
    NVIDIA Video Codec SDK を 13.0 にアップデート
    torikizi authored Jun 25, 2025
    Copy the full SHA
    d58ee4d View commit details
  7. Copy the full SHA
    bb9aaf7 View commit details

Commits on Jul 2, 2025

  1. buildbase.py 更新

    melpon committed Jul 2, 2025
    Copy the full SHA
    33a364b View commit details
  2. Copy the full SHA
    95f51bb View commit details

Commits on Jul 3, 2025

  1. Copy the full SHA
    9b5daf2 View commit details
  2. Copy the full SHA
    8b43afa View commit details
  3. Merge pull request #205 from shiguredo/feature/update-webrtc-m138.720…

    …4.0.1
    
    WEBRTC_BUILD_VERSION を m138.7204.0.1 に上げる
    torikizi authored Jul 3, 2025
    Copy the full SHA
    85d3ad1 View commit details
  4. Copy the full SHA
    4e0b039 View commit details

Commits on Jul 7, 2025

  1. README を更新

    torikizi committed Jul 7, 2025
    Copy the full SHA
    771914b View commit details
  2. Merge pull request #206 from shiguredo/feature/fix-readme-doc

    README を更新
    torikizi authored Jul 7, 2025
    Copy the full SHA
    ebedac2 View commit details
  3. Copy the full SHA
    b46f512 View commit details
  4. Copy the full SHA
    67b6855 View commit details

Commits on Jul 8, 2025

  1. Copy the full SHA
    1e78f87 View commit details
  2. FAQ の説明を修正

    torikizi committed Jul 8, 2025
    Copy the full SHA
    8c359ff View commit details
  3. ドライバーに統一

    torikizi committed Jul 8, 2025
    Copy the full SHA
    dec2e21 View commit details
  4. CHANGES 更新

    melpon committed Jul 8, 2025
    Copy the full SHA
    561d96b View commit details
  5. Merge pull request #208 from shiguredo/feature/fix-amd-amf-autoscaling

    AMF がオートスケール時に正しく動作しないのを修正する
    melpon authored Jul 8, 2025
    Copy the full SHA
    6f84883 View commit details
Showing with 4,073 additions and 1,268 deletions.
  1. +1 −0 .github/copilot-instructions.md
  2. +37 −15 .github/workflows/build.yml
  3. +41 −0 .github/workflows/claude.yml
  4. +105 −0 .github/workflows/formatter.yml
  5. +2 −2 .vscode/c_cpp_properties.json
  6. +78 −5 CHANGES.md
  7. +16 −0 CLAUDE.md
  8. +9 −3 CMakeLists.txt
  9. +5 −5 README.md
  10. +4 −5 VERSION
  11. +291 −1 buildbase.py
  12. +2 −2 doc/development.md
  13. +6 −0 doc/faq.md
  14. +0 −8 doc/known_issues.md
  15. +4 −4 examples/VERSION
  16. +1 −0 examples/messaging_recvonly_sample/macos_arm64/run.py
  17. +32 −7 examples/messaging_recvonly_sample/src/messaging_recvonly_sample.cpp
  18. +1 −0 examples/messaging_recvonly_sample/ubuntu-22.04_x86_64/run.py
  19. +1 −0 examples/messaging_recvonly_sample/ubuntu-24.04_armv8/run.py
  20. +1 −0 examples/messaging_recvonly_sample/ubuntu-24.04_x86_64/run.py
  21. +1 −0 examples/messaging_recvonly_sample/windows_x86_64/run.py
  22. +2 −4 examples/sdl_sample/README.md
  23. +4 −4 examples/sdl_sample/macos_arm64/CMakeLists.txt
  24. +8 −7 examples/sdl_sample/macos_arm64/run.py
  25. +58 −29 examples/sdl_sample/src/sdl_renderer.cpp
  26. +8 −6 examples/sdl_sample/src/sdl_renderer.h
  27. +36 −15 examples/sdl_sample/src/sdl_sample.cpp
  28. +4 −4 examples/sdl_sample/ubuntu-22.04_x86_64/CMakeLists.txt
  29. +8 −7 examples/sdl_sample/ubuntu-22.04_x86_64/run.py
  30. +4 −4 examples/sdl_sample/ubuntu-24.04_armv8/CMakeLists.txt
  31. +8 −7 examples/sdl_sample/ubuntu-24.04_armv8/run.py
  32. +4 −4 examples/sdl_sample/ubuntu-24.04_x86_64/CMakeLists.txt
  33. +8 −7 examples/sdl_sample/ubuntu-24.04_x86_64/run.py
  34. +4 −4 examples/sdl_sample/windows_x86_64/CMakeLists.txt
  35. +8 −7 examples/sdl_sample/windows_x86_64/run.py
  36. +2 −4 examples/sumomo/README.md
  37. +11 −5 examples/sumomo/macos_arm64/CMakeLists.txt
  38. +8 −7 examples/sumomo/macos_arm64/run.py
  39. +105 −0 examples/sumomo/src/ansi_renderer.cpp
  40. +32 −0 examples/sumomo/src/ansi_renderer.h
  41. +330 −0 examples/sumomo/src/base_renderer.cpp
  42. +113 −0 examples/sumomo/src/base_renderer.h
  43. +64 −281 examples/sumomo/src/sdl_renderer.cpp
  44. +13 −72 examples/sumomo/src/sdl_renderer.h
  45. +212 −0 examples/sumomo/src/sixel_renderer.cpp
  46. +33 −0 examples/sumomo/src/sixel_renderer.h
  47. +121 −33 examples/sumomo/src/sumomo.cpp
  48. +11 −5 examples/sumomo/ubuntu-22.04_x86_64/CMakeLists.txt
  49. +8 −7 examples/sumomo/ubuntu-22.04_x86_64/run.py
  50. +11 −5 examples/sumomo/ubuntu-24.04_armv8/CMakeLists.txt
  51. +8 −7 examples/sumomo/ubuntu-24.04_armv8/run.py
  52. +11 −5 examples/sumomo/ubuntu-24.04_x86_64/CMakeLists.txt
  53. +8 −7 examples/sumomo/ubuntu-24.04_x86_64/run.py
  54. +11 −5 examples/sumomo/windows_x86_64/CMakeLists.txt
  55. +8 −7 examples/sumomo/windows_x86_64/run.py
  56. +8 −9 include/sora/aligned_encoder_adapter.h
  57. +6 −3 include/sora/audio_device_module.h
  58. +0 −1 include/sora/audio_output_helper.h
  59. +4 −0 include/sora/camera_device_capturer.h
  60. +6 −2 include/sora/data_channel.h
  61. +8 −8 include/sora/device_video_capturer.h
  62. +1 −1 include/sora/dyn/dyn.h
  63. +1 −0 include/sora/dyn/nvcuvid.h
  64. +1 −0 include/sora/hwenc_amf/amf_video_codec.h
  65. +1 −0 include/sora/hwenc_amf/amf_video_decoder.h
  66. +1 −1 include/sora/hwenc_amf/amf_video_encoder.h
  67. +2 −1 include/sora/hwenc_nvcodec/nvcodec_decoder_cuda.h
  68. +9 −0 include/sora/hwenc_nvcodec/nvcodec_v4l2_capturer.h
  69. +3 −2 include/sora/hwenc_nvcodec/nvcodec_video_decoder.h
  70. +0 −1 include/sora/hwenc_nvcodec/nvcodec_video_encoder.h
  71. +2 −0 include/sora/hwenc_vpl/vpl_video_codec.h
  72. +0 −1 include/sora/hwenc_vpl/vpl_video_decoder.h
  73. +6 −0 include/sora/i420_encoder_adapter.h
  74. +0 −1 include/sora/open_h264_video_codec.h
  75. +1 −1 include/sora/open_h264_video_decoder.h
  76. +1 −1 include/sora/open_h264_video_encoder.h
  77. +6 −3 include/sora/rtc_stats.h
  78. +4 −3 include/sora/scalable_track_source.h
  79. +4 −0 include/sora/session_description.h
  80. +7 −2 include/sora/sora_client_context.h
  81. +30 −12 include/sora/sora_signaling.h
  82. +8 −7 include/sora/sora_video_codec.h
  83. +7 −0 include/sora/sora_video_codec_factory.h
  84. +4 −0 include/sora/sora_video_decoder_factory.h
  85. +5 −0 include/sora/sora_video_encoder_factory.h
  86. +1 −0 include/sora/ssl_verifier.h
  87. +5 −4 include/sora/v4l2/v4l2_video_capturer.h
  88. +9 −1 include/sora/websocket.h
  89. +3 −0 include/sora/zlib_helper.h
  90. +11 −0 multistrap/ubuntu-22.04_armv8.conf
  91. +300 −46 run.py
  92. +10 −1 src/aligned_encoder_adapter.cpp
  93. +4 −1 src/amf_context_impl.cpp
  94. +3 −1 src/amf_context_impl.h
  95. +2 −2 src/android/android_capturer.cpp
  96. +11 −7 src/audio_device_module.cpp
  97. +2 −0 src/audio_output_helper.cpp
  98. +4 −0 src/camera_device_capturer.cpp
  99. +11 −3 src/cuda_context_cuda.cpp
  100. +3 −0 src/cuda_context_cuda.h
  101. +20 −0 src/data_channel.cpp
  102. +5 −3 src/default_video_formats.cpp
  103. +26 −19 src/device_list.cpp
  104. +13 −1 src/device_video_capturer.cpp
  105. +10 −3 src/hwenc_amf/amf_video_codec.cpp
  106. +38 −9 src/hwenc_amf/amf_video_decoder.cpp
  107. +35 −12 src/hwenc_amf/amf_video_encoder.cpp
  108. +14 −0 src/hwenc_nvcodec/nvcodec_decoder_cuda.cpp
  109. +15 −2 src/hwenc_nvcodec/nvcodec_v4l2_capturer.cpp
  110. +8 −4 src/hwenc_nvcodec/nvcodec_video_codec.cpp
  111. +7 −0 src/hwenc_nvcodec/nvcodec_video_codec_cuda.cpp
  112. +3 −0 src/hwenc_nvcodec/nvcodec_video_codec_cuda.h
  113. +15 −3 src/hwenc_nvcodec/nvcodec_video_decoder.cpp
  114. +39 −34 src/hwenc_nvcodec/nvcodec_video_encoder.cpp
  115. +10 −1 src/hwenc_nvcodec/nvcodec_video_encoder_cuda.cpp
  116. +3 −0 src/hwenc_nvcodec/nvcodec_video_encoder_cuda.h
  117. +4 −0 src/hwenc_vpl/vpl_utils.h
  118. +13 −2 src/hwenc_vpl/vpl_video_codec.cpp
  119. +20 −7 src/hwenc_vpl/vpl_video_decoder.cpp
  120. +74 −14 src/hwenc_vpl/vpl_video_encoder.cpp
  121. +10 −1 src/i420_encoder_adapter.cpp
  122. +45 −30 src/mac/mac_audio_output_helper.mm
  123. +25 −14 src/mac/mac_capturer.mm
  124. +1 −1 src/mac/mac_version.mm
  125. +1 −1 src/mac/mac_video_factory.mm
  126. +8 −3 src/open_h264_video_codec.cpp
  127. +13 −2 src/open_h264_video_decoder.cpp
  128. +30 −12 src/open_h264_video_encoder.cpp
  129. +11 −1 src/rtc_ssl_verifier.cpp
  130. +7 −0 src/rtc_stats.cpp
  131. +7 −3 src/scalable_track_source.cpp
  132. +12 −0 src/session_description.cpp
  133. +16 −19 src/sora_client_context.cpp
  134. +6 −0 src/sora_peer_connection_factory.cpp
  135. +44 −1 src/sora_signaling.cpp
  136. +18 −1 src/sora_video_codec.cpp
  137. +19 −5 src/sora_video_codec_factory.cpp
  138. +12 −6 src/sora_video_decoder_factory.cpp
  139. +12 −7 src/sora_video_encoder_factory.cpp
  140. +14 −9 src/ssl_verifier.cpp
  141. +3 −0 src/url_parts.cpp
  142. +18 −6 src/v4l2/v4l2_video_capturer.cpp
  143. +2 −1 src/version.cpp
  144. +9 −2 src/vpl_session_impl.cpp
  145. +2 −2 src/vpl_session_impl.h
  146. +40 −2 src/websocket.cpp
  147. +9 −0 src/zlib_helper.cpp
  148. +1 −1 test/android/app/build.gradle
  149. +17 −0 test/blend2d_iwyu.h
  150. +29 −8 test/connect_disconnect.cpp
  151. +26 −8 test/datachannel.cpp
  152. +4 −3 test/device_list.cpp
  153. +22 −11 test/e2e.cpp
  154. +9 −3 test/fake_video_capturer.cpp
  155. +3 −0 test/fake_video_capturer.h
  156. +25 −4 test/hello.cpp
  157. +21 −3 test/hello.h
  158. +0 −2 test/ios/hello/AppDelegate.h
  159. +1 −2 test/ios/hello/SceneDelegate.h
  160. +0 −1 test/ios/hello/ViewController.h
  161. +192 −74 third_party/NvCodec/NvCodec/NvDecoder/NvDecoder.cpp
  162. +22 −4 third_party/NvCodec/NvCodec/NvDecoder/NvDecoder.h
  163. +120 −21 third_party/NvCodec/NvCodec/NvEncoder/NvEncoder.cpp
  164. +28 −10 third_party/NvCodec/NvCodec/NvEncoder/NvEncoder.h
  165. +0 −2 third_party/NvCodec/NvCodec/NvEncoder/NvEncoderCuda.h
  166. +83 −19 third_party/NvCodec/Utils/NvCodecUtils.h
  167. +53 −32 third_party/NvCodec/include/cuviddec.h
  168. +196 −58 third_party/NvCodec/include/nvEncodeAPI.h
  169. +18 −17 third_party/NvCodec/include/nvcuvid.h
1 change: 1 addition & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- 日本語で反応して
52 changes: 37 additions & 15 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -54,7 +54,7 @@ jobs:
key: windows-cuda-${{ steps.versions.outputs.cuda_version }}.v1
- run: echo "${CUDA_VERSION}" > _install\windows_x86_64\release\cuda.version
if: steps.cache-cuda.outputs.cache-hit == 'true'
- run: python3 run.py --test --run-e2e-test --package ${{ matrix.name }}
- run: python3 run.py build --test --run-e2e-test --package ${{ matrix.name }}
env:
SORA_CPP_SDK_TEMP_DIR: C:\
- name: Get package name
@@ -116,7 +116,7 @@ jobs:
run: |
echo "user=`users`" >> $GITHUB_OUTPUT
id: env
- run: python3 run.py --test --run-e2e-test --package ${{ matrix.name }}
- run: python3 run.py build --test --run-e2e-test --package ${{ matrix.name }}
- name: Get package name
run: |
source _package/${{ matrix.name }}/release/sora.env
@@ -167,6 +167,10 @@ jobs:
runs-on: ubuntu-24.04
os: ubuntu
arch: x86_64
- name: ubuntu-22.04_armv8
runs-on: ubuntu-22.04
os: ubuntu
arch: armv8
- name: ubuntu-24.04_armv8
runs-on: ubuntu-24.04
os: ubuntu
@@ -267,7 +271,7 @@ jobs:
- name: Setup Android SDK
uses: android-actions/setup-android@v3
if: matrix.platform.os == 'android'
- run: python3 run.py --test --run-e2e-test --package ${{ matrix.platform.name }}
- run: python3 run.py build --test --run-e2e-test --package ${{ matrix.platform.name }}
- name: Get package name
run: |
source _package/${{ matrix.platform.name }}/release/sora.env
@@ -297,13 +301,13 @@ jobs:
python3 $app/${{ matrix.platform.name }}/run.py --local-sora-cpp-sdk-dir ..
cp _build/${{ matrix.platform.name }}/release/$app/$app examples_${{ matrix.platform.name }}
done
if: matrix.platform.os == 'ubuntu'
if: matrix.platform.os == 'ubuntu' && matrix.platform.name != 'ubuntu-22.04_armv8'
- name: Upload Examples Artifact
uses: actions/upload-artifact@v4
with:
name: examples_${{ matrix.platform.name }}
path: examples/examples_${{ matrix.platform.name }}
if: matrix.platform.os == 'ubuntu'
if: matrix.platform.os == 'ubuntu' && matrix.platform.name != 'ubuntu-22.04_armv8'

create-release:
name: Create Release
@@ -330,6 +334,9 @@ jobs:
- uses: ./.github/actions/download
with:
platform: ubuntu-24.04_x86_64
- uses: ./.github/actions/download
with:
platform: ubuntu-22.04_armv8
- uses: ./.github/actions/download
with:
platform: ubuntu-24.04_armv8
@@ -338,15 +345,27 @@ jobs:
platform: android
- name: Env to output
run: |
echo "package_paths<<EOF" >> $GITHUB_OUTPUT
cat package_paths.env >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
# package_paths.env の内容を空白区切りに変換
PACKAGE_PATHS=$(tr '\n' ' ' < package_paths.env)
echo "package_paths=${PACKAGE_PATHS}" >> $GITHUB_OUTPUT
id: env
- name: Release (Prerelease)
if: contains(github.ref, 'canary')
run: |
gh release create "${{ github.ref_name }}" \
--prerelease \
--title "${{ github.ref_name }}" \
${{ steps.env.outputs.package_paths }}
env:
GH_TOKEN: ${{ github.token }}
- name: Release
uses: softprops/action-gh-release@v2
with:
files: ${{ steps.env.outputs.package_paths }}
prerelease: ${{ contains(github.ref, 'canary') }}
if: ${{ !contains(github.ref, 'canary') }}
run: |
gh release create "${{ github.ref_name }}" \
--title "${{ github.ref_name }}" \
${{ steps.env.outputs.package_paths }}
env:
GH_TOKEN: ${{ github.token }}

build-windows-examples:
needs: create-release
@@ -485,9 +504,12 @@ jobs:
with:
name: ${{ matrix.example }}_${{ matrix.archive }}
- name: Release Examples
uses: softprops/action-gh-release@v2
with:
files: ${{ matrix.example }}_${{ matrix.archive }}
run: |
gh release upload "${{ github.ref_name }}" \
"${{ matrix.example }}_${{ matrix.archive }}" \
--clobber
env:
GH_TOKEN: ${{ github.token }}

notification:
name: Slack Notification
41 changes: 41 additions & 0 deletions .github/workflows/claude.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Claude Assistant

on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]

env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_SORA_OSS_SONNET }}

jobs:
claude-opus-response-voluntas:
if: github.event.comment.user.login == 'voluntas'
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: anthropics/claude-code-action@beta
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN_VOLUNTAS }}
github_token: ${{ secrets.GITHUB_TOKEN }}
model: "claude-opus-4-20250514"
trigger_phrase: "!opus"
timeout_minutes: 15
disallowed_tools: "Bash(git:*)"

claude-sonnet-response-others:
if: contains(fromJSON('["Hexa", "akko", "sile", "melpon", "tnamao", "torikizi", "miosakuma", "zztkm", "t-miya"]'), github.event.comment.user.login)
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: anthropics/claude-code-action@beta
with:
anthropic_api_key: ${{ env.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
model: "claude-sonnet-4-20250514"
trigger_phrase: "!sonnet"
timeout_minutes: 15
disallowed_tools: "Bash(git:*)"
105 changes: 105 additions & 0 deletions .github/workflows/formatter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: formatter

on:
push:
paths-ignore:
- "doc/**"
- "**.md"
- "LICENSE"
- "NOTICE"

jobs:
run-ubuntu:
strategy:
fail-fast: false
matrix:
platform:
- name: ubuntu-24.04_x86_64
runs-on: ubuntu-24.04
os: ubuntu
arch: x86_64
name: Build sora-cpp-sdk for ${{ matrix.platform.name }}
runs-on: ${{ matrix.platform.runs-on }}
env:
TEST_SIGNALING_URL: ${{ secrets.TEST_SIGNALING_URL }}
TEST_CHANNEL_ID_PREFIX: ${{ secrets.TEST_CHANNEL_ID_PREFIX }}
TEST_SECRET_KEY: ${{ secrets.TEST_SECRET_KEY }}
TEST_MATRIX_NAME: ${{ matrix.platform.name }}
steps:
- uses: actions/checkout@v4
# IWYU を適用するにはまずビルドする必要がある
# Ubuntu 24.04 だと libtinfo5 が見つからない問題があるので、その修正
# ref: https://qiita.com/gengen16k/items/88cf3c18a40a94205fab
- name: Fix CUDA issues for Ubuntu 24.04
if: matrix.platform.name == 'ubuntu-24.04_x86_64'
run: |
sudo tee /etc/apt/sources.list.d/jammy.list << EOF
deb http://archive.ubuntu.com/ubuntu/ jammy universe
EOF
sudo tee /etc/apt/preferences.d/pin-jammy <<EOF
Package: *
Pin: release n=jammy
Pin-Priority: -10
Package: libtinfo5
Pin: release n=jammy
Pin-Priority: 990
EOF
- name: Install deps for ${{ matrix.platform.name }}
if: matrix.platform.os == 'ubuntu' && matrix.platform.arch == 'x86_64'
run: |
source VERSION
sudo apt-get update
sudo apt-get install -y software-properties-common
# X11
sudo apt-get install libx11-dev libxext-dev
# OpenGL
sudo apt-get install -y libgl-dev
# CUDA
# libssl1.1 が必要
wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.0g-2ubuntu4_amd64.deb
sudo dpkg -i libssl1.1_1.1.0g-2ubuntu4_amd64.deb
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_*all.deb
sudo apt-get update
DEBIAN_FRONTEND=noninteractive sudo apt-get -y install cuda=$CUDA_VERSION
# clang-20
wget https://apt.llvm.org/llvm.sh
chmod a+x llvm.sh
sudo ./llvm.sh 20
- run: python3 run.py build --test ${{ matrix.platform.name }}
- name: Build Examples
run: |
cd examples
mkdir examples_${{ matrix.platform.name }}
for app in sdl_sample sumomo messaging_recvonly_sample; do
python3 $app/${{ matrix.platform.name }}/run.py --local-sora-cpp-sdk-dir ..
done
if: matrix.platform.os == 'ubuntu' && matrix.platform.name != 'ubuntu-22.04_armv8'
- name: Run IWYU
run: python3 run.py iwyu ubuntu-24.04_x86_64 || git diff --exit-code
- name: Run clang format
run: python3 run.py format || git diff --exit-code
- name: Check diff
run: git diff --exit-code

notification:
name: Slack Notification
runs-on: ubuntu-24.04
needs:
- run-ubuntu
if: always()
steps:
- uses: rtCamp/action-slack-notify@v2
if: |
needs.run-ubuntu.result == 'failure'
env:
SLACK_CHANNEL: sora-cpp-sdk
SLACK_COLOR: danger
SLACK_TITLE: Build failed
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
4 changes: 2 additions & 2 deletions .vscode/c_cpp_properties.json
Original file line number Diff line number Diff line change
@@ -53,7 +53,7 @@
"${workspaceFolder}/_install/ubuntu-24.04_x86_64/release/amf/amf",
"${workspaceFolder}/_install/ubuntu-24.04_x86_64/release/blend2d/include",
"${workspaceFolder}/examples/_install/ubuntu-24.04_x86_64/release/cli11/include",
"${workspaceFolder}/examples/_install/ubuntu-24.04_x86_64/release/sdl2/include",
"${workspaceFolder}/examples/_install/ubuntu-24.04_x86_64/release/sdl3/include",
"${workspaceFolder}/_build/ubuntu-24.04_x86_64/release/sora",
"${workspaceFolder}/third_party/NvCodec/include",
"${workspaceFolder}/third_party/NvCodec/NvCodec",
@@ -93,7 +93,7 @@
"${workspaceFolder}/_install/ubuntu-24.04_x86_64/debug/amf/amf",
"${workspaceFolder}/_install/ubuntu-24.04_x86_64/debug/blend2d/include",
"${workspaceFolder}/examples/_install/ubuntu-24.04_x86_64/debug/cli11/include",
"${workspaceFolder}/examples/_install/ubuntu-24.04_x86_64/debug/sdl2/include",
"${workspaceFolder}/examples/_install/ubuntu-24.04_x86_64/debug/sdl3/include",
"${workspaceFolder}/_build/ubuntu-24.04_x86_64/debug/sora",
"${workspaceFolder}/third_party/NvCodec/include",
"${workspaceFolder}/third_party/NvCodec/NvCodec",
83 changes: 78 additions & 5 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -11,17 +11,66 @@

## develop

- [CHANGE] ライブラリのヘッダーファイルとソースファイルで使用されている include を整理する
- 今まで間接的にインクルードされていたファイルが含まれなくなっている可能性があるので、破壊的変更になる
- @melpon
- [CHANGE] run.py のビルドをサブコマンド化する
- 今まで `python3 run.py ubuntu-24.04_x86_64` でビルドしていたコマンドが`python3 run.py build ubuntu-24.04_x86_64` となる
- @melpon
- [ADD] clang-include-cleaner や clang-format を実行するサブコマンドを追加する
- `python3 run.py iwyu ubuntu-24.04_x86_64`
- `python3 run.py format`
- なお iwyu を実行するには事前にビルドしておく必要がある
- @melpon
- [ADD] clang-include-cleaner や clang-format を GitHub Actions で実行し、差分があったらエラーにするワークフローを追加
- @melpon
- [ADD] Intel VPL の VP9 エンコーダを動くようにする
- @melpon

### misc

- [CHANGE] SDL サンプルと Sumomo から `--multistream` オプションを削除する
- マルチストリーム機能は Sora サーバー側で制御されるため、クライアント側でオプションは不要となりました
- @torikizi
- [UPDATE] SDL2 から SDL3 にアップデートする
- SDL3 の API 変更に伴い、sdl_sample と sumomo で以下の修正を実施:
- ヘッダーインクルードを `<SDL2/SDL.h>` から `<SDL3/SDL.h>` に変更
- `SDL2::SDL2main` をリンクする代わりに `SDL3/SDL_main.h` のインクルードを追加
- `SDL_Init` の戻り値チェックを bool 型に変更
- `SDL_PollEvent` の戻り値チェックを bool 型に変更
- `SDL_CreateWindow` の位置引数を削除
- `SDL_RENDERER_ACCELERATED` フラグを削除
- `SDL_SetWindowFullscreen` の引数を bool 型に変更
- `SDL_ShowCursor` / `SDL_HideCursor` を分離された関数に変更
- イベント名を SDL3 形式に変更(例: `SDL_WINDOWEVENT``SDL_EVENT_WINDOW_RESIZED`
- キーコードを大文字に変更(例: `SDLK_f``SDLK_F`
- `SDL_KeyboardEvent``keysym.sym``key` に変更
- `SDL_RenderCopy``SDL_RenderTexture` に変更
- `SDL_Rect``SDL_FRect` に変更(座標を float 型に)
- `SDL_CreateRGBSurfaceFrom``SDL_CreateSurfaceFrom` に変更
- `SDL_FreeSurface``SDL_DestroySurface` に変更
- @melpon
- [ADD] sumomo のレンダラーに Sixel と ANSI エスケープシーケンスでの表示を追加する
- `--use-sixel`, `--sixel-width`, `--sixel-height` オプションの追加
- `--use-ansi`, `--ansi-width`, `--ansi-height` オプションの追加
- また、この過程で SDLRenderer の SDL に依存しない部分をベースクラス(BaseRenderer)としてまとめた
- @melpon

## 2025.4.0

**リリース日**: 2025-07-09

- [CHANGE] OnDataChannel コールバックで、`#` で始まっていないラベルも Open 状態になったことを通知する
- @melpon
- [CHANGE] Android ビルドに利用するコンパイラを Android NDK に内包されている clang ではなく、libwebrtc の clang に変更する
- @melpon
- [ADD] rpc ラベルにメッセージが来た時に OnRpc コールバックを呼び出す
- @melpon
- [UPDATE] libwebrtc を m137.7151.0.0 にあげる
- [UPDATE] libwebrtc を m138.7204.0.1 にあげる
- `rtc::revive::` を `webrtc::revive::` に変更する
- m137 で `rtc_base/third_party/base64/base64.h` が削除されたため、websocket.cpp で `rtc_base/base64.h` をインクルードするように変更する
- `webrtc::Base64::Encode` から `webrtc::Base64Encode` に変更する
- Android ビルドに利用するコンパイラを Android NDK に内包されている clang ではなく、libwebrtc の clang に変更する
- `sora::AudioDeviceModuleConfig` から `task_queue_factory` を削除して `env` を追加する
- `webrtc::CreateOpenSLESAudioDeviceModule``webrtc::CreateJavaAudioDeviceModule` に変更する
- @miosakuma @torikizi @melpon
- [UPDATE] Android NDK を r28b にあげる
- 16KB ページサイズに対応するため
@@ -33,15 +82,38 @@
- @melpon
- [UPDATE] Android SDK Command-line tools のバージョンを 13114758 にあげる
- @melpon
- [UPDATE] `NVIDIA Video Codec SDK`[13.0](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/index.html) にアップデートする
- NVIDIA Video Codec SDK で新たに追加された `NvEncOutputFrame` 構造体に対応する
- `v_packet_``std::vector<std::vector<uint8_t>>` から `std::vector<NvEncOutputFrame>` に変更する
- `for (std::vector<uint8_t>& packet : v_packet_)``for (NvEncOutputFrame& output : v_packet_)` に変更する
- ループ内に `std::vector<uint8_t>& packet = output.frame;` を追加し、既存処理との互換性を維持する
- コーデックごとに実行していたキーフレーム判定を NvEncOutputFrame のフレーム情報を利用して行うように変更する
- @torikizi
- [UPDATE] Blend2D を公式サイトからダウンロードするように変更する
- 今までは現時点の master のコミットハッシュを使っていたが、asmjit と一緒にバージョンを管理しないといけなかった
- 公式サイトからのダウンロードだと asmjit を内包してるのでこっちの方が管理が楽そうという判断
- @melpon
- [ADD] ubuntu-22.04_armv8 を追加
- @melpon
- [ADD] rpc ラベルにメッセージが来た時に OnRpc コールバックを呼び出す
- @melpon
- [ADD] run.py の引数に `--disable-cuda` を追加
- @melpon
- [ADD] インストールするファイルに cmake/android.toolchain.cmake を追加
- @melpon
- [FIX] AMD AMF のデコーダーがオートスケーリング時に映像がおかしくなるのを修正
- @melpon

### misc

- [UPDATE] CMake を 4.0.2 にあげる
- @voluntas
- [UPDATE] CMake を 4.0.3 にあげる
- @voluntas @torikizi
- [UPDATE] SDL を 2.32.6 に上げる
- @voluntas
- [UPDATE] test と examples の `rtc::``webrtc::` に変更する
- @torikizi
- [ADD] .github ディレクトリに copilot-instructions.md を追加
- @torikizi

## 2025.3.1

@@ -52,6 +124,7 @@
- Sora から type: switched のメッセージを受け取って WS を切断するタイミングと、ユーザーからの切断のタイミングが被るとクラッシュすることがある
- WS を非同期処理で Close しているが、この処理中にユーザーからの Disconnect 呼び出しをすると、WS の Close 完了前に WS が破棄されてしまって未定義動作となる
- WS を綺麗にシャットダウンするのを諦めて(Close フレームを送らず)、同期的に TCP ソケットを閉じるように修正する
- @melpon

## 2025.3.0

16 changes: 16 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# CLAUDE.md

- 一切忖度しないこと
- 常に日本語を利用すること

## レビューについて

- レビューはかなり厳しくすること
- レビューの表現は、シンプルにすること
- レビューの表現は、日本語で行うこと
- レビューの表現は、指摘内容を明確にすること
- レビューの表現は、指摘内容を具体的にすること
- レビューの表現は、指摘内容を優先順位をつけること
- レビューの表現は、指摘内容を優先順位をつけて、重要なものから順に記載すること
- ドキュメントは別に書いているので、ドキュメトに付いては考慮しないこと
- 変更点とリリースノートの整合性を確認すること
12 changes: 9 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -39,17 +39,22 @@ elseif (SORA_TARGET STREQUAL "ubuntu-22.04_x86_64")
set(SORA_TARGET_OS "ubuntu")
set(SORA_TARGET_OS_VERSION "22.04")
set(SORA_TARGET_ARCH "x86_64")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU_2204")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU")
elseif (SORA_TARGET STREQUAL "ubuntu-24.04_x86_64")
set(SORA_TARGET_OS "ubuntu")
set(SORA_TARGET_OS_VERSION "24.04")
set(SORA_TARGET_ARCH "x86_64")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU_2404")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU")
elseif (SORA_TARGET STREQUAL "ubuntu-22.04_armv8")
set(SORA_TARGET_OS "ubuntu")
set(SORA_TARGET_OS_VERSION "22.04")
set(SORA_TARGET_ARCH "armv8")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU")
elseif (SORA_TARGET STREQUAL "ubuntu-24.04_armv8")
set(SORA_TARGET_OS "ubuntu")
set(SORA_TARGET_OS_VERSION "24.04")
set(SORA_TARGET_ARCH "armv8")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU_2404")
set(SORA_TARGET_DEF "SORA_CPP_SDK_UBUNTU")
endif()

list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake)
@@ -584,6 +589,7 @@ install(EXPORT sora-config
install(
FILES
cmake/FindWebRTC.cmake
cmake/android.toolchain.cmake
DESTINATION ${CMAKE_INSTALL_DATADIR}/cmake
)
install(FILES LICENSE NOTICE.md
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ Please read <https://github.com/shiguredo/oss/blob/master/README.en.md> before u
- 各プラットフォームで利用可能な HWA への対応
- [Intel VPL](https://github.com/intel/libvpl)
- VP9 / AV1 / H.264 / H.265
- VP9 はエンコードのみの対応です、詳細は [既知の問題](doc/known_issues.md) をご確認ください
- VP9 はエンコードのみの対応です
- [NVIDIA Video Codec SDK](https://developer.nvidia.com/video-codec-sdk)
- VP8 / VP9 / AV1 / H.264 / H.265
- VP8 / VP9 はチップセットが対応していないため、デコードのみの対応です
@@ -53,7 +53,7 @@ _hololens2 は無視してください_

## 対応 Sora

- WebRTC SFU Sora 2024.2.0 以降
- WebRTC SFU Sora 2025.1.0 以降

## 動作環境

@@ -71,7 +71,7 @@ _hololens2 は無視してください_

## 既知の問題

[known_issues.md](doc/known_issues.md) をお読みください
[known_issues.md](doc/known_issues.md) をご確認ください

## FAQ

@@ -146,10 +146,10 @@ limitations under the License.

## NVIDIA Video Codec SDK

<https://docs.nvidia.com/video-technologies/video-codec-sdk/12.2/index.html>
<https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/index.html>


<https://docs.nvidia.com/video-technologies/video-codec-sdk/12.2/license/index.html>
<https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/license/index.html>

```text
“This software contains source code provided by NVIDIA Corporation.”
9 changes: 4 additions & 5 deletions VERSION
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
SORA_CPP_SDK_VERSION=2025.4.0-canary.1
WEBRTC_BUILD_VERSION=m137.7151.3.0
SORA_CPP_SDK_VERSION=2025.5.0-canary.0
WEBRTC_BUILD_VERSION=m138.7204.0.1
BOOST_VERSION=1.88.0
CMAKE_VERSION=4.0.2
CMAKE_VERSION=4.0.3
CUDA_VERSION=11.8.0-1
ANDROID_NDK_VERSION=r28b
ANDROID_NATIVE_API_LEVEL=29
ANDROID_SDK_CMDLINE_TOOLS_VERSION=13114758
VPL_VERSION=v2.15.0
AMF_VERSION=v1.4.36
OPENH264_VERSION=v2.6.0
BLEND2D_VERSION=717cbf4bc0f2ca164cf2f0c48f0497779241b6c5
ASMJIT_VERSION=e8c8e2e48a1a38154c8e8864eb3bc61db80a1e31
BLEND2D_VERSION=0.12.0
CATCH2_VERSION=v3.8.1
292 changes: 291 additions & 1 deletion buildbase.py
Original file line number Diff line number Diff line change
@@ -378,6 +378,7 @@ def git_clone_shallow(url, hash, dir, submodule=False):


def apply_patch(patch, dir, depth):
patch = os.path.abspath(patch)
with cd(dir):
logging.info(f"patch -p{depth} < {patch}")
if platform.system() == "Windows":
@@ -522,6 +523,7 @@ class WebrtcInfo(NamedTuple):
webrtc_library_dir: str
clang_dir: str
libcxx_dir: str
libcxxabi_dir: str


def get_webrtc_info(
@@ -538,6 +540,9 @@ def get_webrtc_info(
webrtc_library_dir=os.path.join(webrtc_install_dir, "lib"),
clang_dir=os.path.join(install_dir, "llvm", "clang"),
libcxx_dir=os.path.join(install_dir, "llvm", "libcxx"),
libcxxabi_dir=os.path.join(
webrtc_install_dir, "include", "third_party", "libc++abi", "src"
),
)
else:
webrtc_build_source_dir = os.path.join(
@@ -558,6 +563,9 @@ def get_webrtc_info(
webrtc_build_source_dir, "src", "third_party", "llvm-build", "Release+Asserts"
),
libcxx_dir=os.path.join(webrtc_build_source_dir, "src", "third_party", "libc++", "src"),
libcxxabi_dir=os.path.join(
webrtc_build_source_dir, "src", "third_party", "libc++abi", "src"
),
)


@@ -926,7 +934,7 @@ def build_sora(
]

with cd(local_sora_cpp_sdk_dir):
cmd(["python3", "run.py", platform, *local_sora_cpp_sdk_args])
cmd(["python3", "run.py", "build", platform, *local_sora_cpp_sdk_args])


class SoraInfo(NamedTuple):
@@ -1220,6 +1228,93 @@ def install_sdl2(
cmd(["cmake", "--install", ".", "--config", configuration])


@versioned
def install_sdl3(
version, source_dir, build_dir, install_dir, debug: bool, platform: str, cmake_args: List[str]
):
url = f"https://github.com/libsdl-org/SDL/releases/download/release-{version}/SDL3-{version}.tar.gz"
path = download(url, source_dir)
sdl3_source_dir = os.path.join(source_dir, "sdl3")
sdl3_build_dir = os.path.join(build_dir, "sdl3")
sdl3_install_dir = os.path.join(install_dir, "sdl3")
rm_rf(sdl3_source_dir)
rm_rf(sdl3_build_dir)
rm_rf(sdl3_install_dir)
extract(path, source_dir, "sdl3")

mkdir_p(sdl3_build_dir)
with cd(sdl3_build_dir):
configuration = "Debug" if debug else "Release"
cmake_args = cmake_args[:]
cmake_args += [
sdl3_source_dir,
f"-DCMAKE_BUILD_TYPE={configuration}",
f"-DCMAKE_INSTALL_PREFIX={cmake_path(sdl3_install_dir)}",
"-DBUILD_SHARED_LIBS=OFF",
"-DSDL_STATIC=ON",
"-DSDL_SHARED=OFF",
]
if platform == "windows":
cmake_args += [
f"-DCMAKE_MSVC_RUNTIME_LIBRARY={'MultiThreaded' if not debug else 'MultiThreadedDebug'}",
"-DSDL_AUDIO=OFF",
"-DSDL_JOYSTICK=OFF",
"-DSDL_HAPTIC=OFF",
# GitHub Actions 上で gameinput.h が存在しないのに
# なぜか check_c_source_compiles() が成功してしまうので
# HAVE_GAMEINPUT_H=0 で強制的に無効化する
"-DHAVE_GAMEINPUT_H=0",
]
elif platform == "macos":
# どの環境でも同じようにインストールされるようにするため全部 ON/OFF を明示的に指定する
cmake_args += [
"-DSDL_AUDIO=OFF",
"-DSDL_VIDEO=ON",
"-DSDL_RENDER=ON",
"-DSDL_HAPTIC=ON",
"-DSDL_POWER=ON",
"-DSDL_JOYSTICK=ON",
"-DSDL_SENSOR=ON",
"-DSDL_OPENGL=ON",
"-DSDL_OPENGLES=ON",
"-DSDL_METAL=ON",
"-DSDL_VULKAN=OFF",
]
elif platform == "linux":
# どの環境でも同じようにインストールされるようにするため全部 ON/OFF を明示的に指定する
cmake_args += [
"-DSDL_AUDIO=OFF",
"-DSDL_VIDEO=ON",
"-DSDL_RENDER=ON",
"-DSDL_HAPTIC=ON",
"-DSDL_POWER=ON",
"-DSDL_JOYSTICK=ON",
"-DSDL_SENSOR=ON",
"-DSDL_OPENGL=ON",
"-DSDL_OPENGLES=ON",
"-DSDL_X11=ON",
"-DSDL_X11_SHARED=OFF",
"-DSDL_X11_XCURSOR=OFF",
"-DSDL_X11_XDBE=OFF",
"-DSDL_X11_XFIXES=OFF",
"-DSDL_X11_XINPUT=OFF",
"-DSDL_X11_XRANDR=OFF",
"-DSDL_X11_XSCRNSAVER=OFF",
"-DSDL_X11_XSHAPE=OFF",
"-DSDL_X11_XSYNC=OFF",
"-DSDL_WAYLAND=OFF",
"-DSDL_VULKAN=OFF",
"-DSDL_KMSDRM=OFF",
"-DSDL_RPI=OFF",
]
cmd(["cmake"] + cmake_args)

cmd(
["cmake", "--build", ".", "--config", configuration, f"-j{multiprocessing.cpu_count()}"]
)
cmd(["cmake", "--install", ".", "--config", configuration])


@versioned
def install_cli11(version, install_dir):
cli11_install_dir = os.path.join(install_dir, "cli11")
@@ -1301,6 +1396,33 @@ def install_vpl(version, configuration, source_dir, build_dir, install_dir, cmak
cmd(["cmake", "--install", ".", "--config", configuration])


@versioned
def install_blend2d_official(
version,
configuration,
source_dir,
build_dir,
install_dir,
ios,
cmake_args,
):
rm_rf(os.path.join(source_dir, "blend2d"))
rm_rf(os.path.join(build_dir, "blend2d"))
rm_rf(os.path.join(install_dir, "blend2d"))

url = f"https://blend2d.com/download/blend2d-{version}.tar.gz"
path = download(url, source_dir)
extract(path, source_dir, "blend2d")
_build_blend2d(
configuration=configuration,
source_dir=source_dir,
build_dir=build_dir,
install_dir=install_dir,
ios=ios,
cmake_args=cmake_args,
)


@versioned
def install_blend2d(
version,
@@ -1326,7 +1448,17 @@ def install_blend2d(
asmjit_version,
os.path.join(source_dir, "blend2d", "3rdparty", "asmjit"),
)
_build_blend2d(
configuration=configuration,
source_dir=source_dir,
build_dir=build_dir,
install_dir=install_dir,
ios=ios,
cmake_args=cmake_args,
)


def _build_blend2d(configuration, source_dir, build_dir, install_dir, ios, cmake_args):
mkdir_p(os.path.join(build_dir, "blend2d"))
with cd(os.path.join(build_dir, "blend2d")):
cmd(
@@ -1741,6 +1873,7 @@ def install_opus(
"cmake",
f"-DCMAKE_INSTALL_PREFIX={cmake_path(opus_install_dir)}",
f"-DCMAKE_BUILD_TYPE={configuration}",
"-DCMAKE_POSITION_INDEPENDENT_CODE=ON",
"-DOPUS_BUILD_SHARED_LIBRARY=OFF",
"-DOPUS_BUILD_TESTING=OFF",
"-DOPUS_BUILD_PROGRAMS=OFF",
@@ -1797,6 +1930,163 @@ def install_amf(version, install_dir):
)


@versioned
def install_mbedtls(version, source_dir, build_dir, install_dir, debug, cmake_args):
rm_rf(os.path.join(source_dir, "mbedtls"))
rm_rf(os.path.join(build_dir, "mbedtls"))
rm_rf(os.path.join(install_dir, "mbedtls"))
git_clone_shallow(
"https://github.com/Mbed-TLS/mbedtls.git",
version,
os.path.join(source_dir, "mbedtls"),
)
with cd(os.path.join(source_dir, "mbedtls")):
configuration = "Debug" if debug else "Release"
cmd(["python3", "scripts/config.py", "set", "MBEDTLS_SSL_DTLS_SRTP"])
cmd(
[
"cmake",
f"-DCMAKE_INSTALL_PREFIX={cmake_path(os.path.join(install_dir, 'mbedtls'))}",
f"-DCMAKE_BUILD_TYPE={configuration}",
"-DCMAKE_POSITION_INDEPENDENT_CODE=ON",
"-B",
os.path.join(build_dir, "mbedtls"),
]
+ cmake_args
)
cmd(
[
"cmake",
"--build",
os.path.join(build_dir, "mbedtls"),
f"-j{multiprocessing.cpu_count()}",
"--config",
"Release",
]
)
cmd(["cmake", "--install", os.path.join(build_dir, "mbedtls")])


@versioned
def install_libjpeg_turbo(version, source_dir, build_dir, install_dir, configuration, cmake_args):
libjpeg_source_dir = os.path.join(source_dir, "libjpeg-turbo")
libjpeg_build_dir = os.path.join(build_dir, "libjpeg-turbo")
libjpeg_install_dir = os.path.join(install_dir, "libjpeg-turbo")
rm_rf(libjpeg_source_dir)
rm_rf(libjpeg_build_dir)
rm_rf(libjpeg_install_dir)
git_clone_shallow(
"https://github.com/libjpeg-turbo/libjpeg-turbo.git",
version,
libjpeg_source_dir,
)
mkdir_p(libjpeg_build_dir)
with cd(libjpeg_build_dir):
cmd(
[
"cmake",
libjpeg_source_dir,
f"-DCMAKE_INSTALL_PREFIX={libjpeg_install_dir}",
f"-DCMAKE_BUILD_TYPE={configuration}",
"-DENABLE_SHARED=FALSE",
"-DENABLE_STATIC=TRUE",
"-DCMAKE_BUILD_TYPE=Release",
"-DCMAKE_POSITION_INDEPENDENT_CODE=ON",
]
+ cmake_args
)
cmd(
[
"cmake",
"--build",
libjpeg_build_dir,
f"-j{multiprocessing.cpu_count()}",
"--config",
"Release",
]
)
cmd(["cmake", "--install", libjpeg_build_dir])


@versioned
def install_libyuv(
version, source_dir, build_dir, install_dir, libjpeg_turbo_dir, configuration, cmake_args
):
libyuv_source_dir = os.path.join(source_dir, "libyuv")
libyuv_build_dir = os.path.join(build_dir, "libyuv")
libyuv_install_dir = os.path.join(install_dir, "libyuv")
rm_rf(libyuv_source_dir)
rm_rf(libyuv_build_dir)
rm_rf(libyuv_install_dir)
git_clone_shallow(
"https://chromium.googlesource.com/libyuv/libyuv",
version,
libyuv_source_dir,
)
mkdir_p(libyuv_build_dir)
with cd(libyuv_build_dir):
cmd(
[
"cmake",
libyuv_source_dir,
f"-DCMAKE_INSTALL_PREFIX={cmake_path(libyuv_install_dir)}",
f"-DCMAKE_BUILD_TYPE={configuration}",
f"-DCMAKE_PREFIX_PATH={cmake_path(libjpeg_turbo_dir)}",
*cmake_args,
]
)
cmd(
[
"cmake",
"--build",
libyuv_build_dir,
f"-j{multiprocessing.cpu_count()}",
"--config",
"Release",
]
)
cmd(["cmake", "--install", libyuv_build_dir])


@versioned
def install_aom(version, source_dir, build_dir, install_dir, configuration, cmake_args):
aom_source_dir = os.path.join(source_dir, "aom")
aom_build_dir = os.path.join(build_dir, "aom")
aom_install_dir = os.path.join(install_dir, "aom")
rm_rf(aom_source_dir)
rm_rf(aom_build_dir)
rm_rf(aom_install_dir)
git_clone_shallow(
"https://aomedia.googlesource.com/aom",
version,
aom_source_dir,
)
with cd(aom_source_dir):
cmd(
[
"cmake",
"-B",
aom_build_dir,
f"-DCMAKE_INSTALL_PREFIX={cmake_path(aom_install_dir)}",
f"-DCMAKE_BUILD_TYPE={configuration}",
"-DENABLE_DOCS=OFF",
"-DENABLE_TESTS=OFF",
*cmake_args,
]
)
cmd(
[
"cmake",
"--build",
aom_build_dir,
f"-j{multiprocessing.cpu_count()}",
"--config",
"Release",
]
)
cmd(["cmake", "--install", aom_build_dir])


class PlatformTarget(object):
def __init__(self, os, osver, arch, extra=None):
self.os = os
4 changes: 2 additions & 2 deletions doc/development.md
Original file line number Diff line number Diff line change
@@ -15,7 +15,7 @@

```bash
# ../webrtc-build に shiguredo-webrtc-build/webrtc-build がある場合
python3 run.py ubuntu-24.04_x86_64 --local-webrtc-build-dir ../webrtc-build
python3 run.py build ubuntu-24.04_x86_64 --local-webrtc-build-dir ../webrtc-build
```

この時、VERSION に指定している WEBRTC_BUILD_VERSION に関係なく、現在 webrtc-build リポジトリでチェックアウトされている内容でビルドするため、バージョンの不整合に注意すること。
@@ -26,7 +26,7 @@ C++ SDK をデバッグビルドするには、libwebrtc も含めて、依存
しかし libwebrtc のバイナリはリリースビルドであるため、libwebrtc のデバッグバイナリを作るにはローカルの webrtc-build を利用する必要がある。

```bash
python3 run.py ubuntu-24.04_x86_64 --debug --local-webrtc-build-dir ../webrtc-build
python3 run.py build ubuntu-24.04_x86_64 --debug --local-webrtc-build-dir ../webrtc-build
```

このように `--debug` を付けると、C++ SDK だけでなく、ローカルの webrtc-build を含む全ての依存ライブラリもデバッグビルドを行う。
6 changes: 6 additions & 0 deletions doc/faq.md
Original file line number Diff line number Diff line change
@@ -47,3 +47,9 @@ Sora C++ SDK は 4K@30fps での映像の配信に対応していますが、
その場合、 `SoraVideoEncoderFactoryConfig` という構造体の `force_i420_conversion` フラグを `false` にすることで、4K@30fps で映像を配信できる場合があります。

かなり内部的な話なので詳細については [コードのコメント](https://github.com/shiguredo/sora-cpp-sdk/blob/c29d3d6ec721ddc3d625275296506750a63460d4/include/sora/sora_video_encoder_factory.h#L59-L73) をご確認ください。

## AMD AMF で映像受信時に映像の停止やデコード失敗が発生する

AMD AMF を使用して映像を受信する際に、映像の停止やデコード失敗が発生することがあります。
その場合は、AMD のグラフィックスドライバーのバージョンを更新することで解決できる可能性があります。
[AMD のサポートページ](https://www.amd.com/ja/support/download/drivers.html)から最新のドライバーをダウンロードしてインストールした後、映像の受信を試してみてください。
8 changes: 0 additions & 8 deletions doc/known_issues.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,5 @@
# 既知の問題

## Intel VPL で VP9 をエンコードするとエラーが発生する

現在 Intel VPL を使用した時に VP9 の映像を送信できません。
そのため Sora C++ SDK では Intel VPL の VP9 HWA を使用したエンコードを無効化しています。
VP9 を使用する場合はソフトウェアエンコードを使用するようになっています。

VPL の VP9 のデコードには問題はないため、受信は利用可能です。

## Safari 18.0 以降で送信した H.265 を Sora C++ SDK 2024.7.0 で受信できない

Safari 18.0 以降のバージョンで送信した H.265 の映像が Sora C++ SDK 2024.7.0 で受信できません。
8 changes: 4 additions & 4 deletions examples/VERSION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
SORA_CPP_SDK_VERSION=2025.4.0-canary.1
WEBRTC_BUILD_VERSION=m137.7151.3.0
SORA_CPP_SDK_VERSION=2025.5.0-canary.0
WEBRTC_BUILD_VERSION=m138.7204.0.1
BOOST_VERSION=1.88.0
CMAKE_VERSION=4.0.2
SDL2_VERSION=2.32.6
CMAKE_VERSION=4.0.3
SDL3_VERSION=3.2.16
CLI11_VERSION=v2.5.0
1 change: 1 addition & 0 deletions examples/messaging_recvonly_sample/macos_arm64/run.py
Original file line number Diff line number Diff line change
@@ -142,6 +142,7 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
Original file line number Diff line number Diff line change
@@ -1,14 +1,39 @@
#include <csignal>
#include <cstdint>
#include <cstdlib>
#include <iostream>
#include <memory>
#include <optional>
#include <ostream>
#include <string>
#include <utility>
#include <vector>

// Boost
#include <boost/asio/executor_work_guard.hpp>
#include <boost/asio/io_context.hpp>
#include <boost/asio/signal_set.hpp>
#include <boost/json/parse.hpp>
#include <boost/json/value.hpp>
#include <boost/json/value_to.hpp>
#include <boost/system/detail/error_code.hpp>

// WebRTC
#include <api/rtp_receiver_interface.h>
#include <api/rtp_transceiver_interface.h>
#include <api/scoped_refptr.h>
#include <rtc_base/logging.h>

// Sora
#include <sora/sora_client_context.h>
#ifdef _WIN32
#include <rtc_base/win/scoped_com_initializer.h>
#endif

// CLI11
#include <CLI/CLI.hpp>

#ifdef _WIN32
#include <rtc_base/win/scoped_com_initializer.h>
#endif
// Sora C++ SDK
#include <sora/sora_client_context.h>
#include <sora/sora_signaling.h>

struct MessagingRecvOnlySampleConfig {
std::string signaling_url;
@@ -92,8 +117,8 @@ class MessagingRecvOnlySample
<< " bytes" << std::endl;
}

void OnTrack(webrtc::scoped_refptr<webrtc::RtpTransceiverInterface> transceiver)
override {}
void OnTrack(webrtc::scoped_refptr<webrtc::RtpTransceiverInterface>
transceiver) override {}
void OnRemoveTrack(
webrtc::scoped_refptr<webrtc::RtpReceiverInterface> receiver) override {}

Original file line number Diff line number Diff line change
@@ -171,6 +171,7 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
Original file line number Diff line number Diff line change
@@ -185,6 +185,7 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
Original file line number Diff line number Diff line change
@@ -171,6 +171,7 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
1 change: 1 addition & 0 deletions examples/messaging_recvonly_sample/windows_x86_64/run.py
Original file line number Diff line number Diff line change
@@ -142,6 +142,7 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
6 changes: 2 additions & 4 deletions examples/sdl_sample/README.md
Original file line number Diff line number Diff line change
@@ -132,13 +132,13 @@ _build/ubuntu-24.04_x86_64/release/sdl_sample/
Windows の場合

```powershell
> .\sdl_sample.exe --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora --multistream true
> .\sdl_sample.exe --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora
```

Windows 以外の場合

```shell
./sdl_sample --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora --multistream true
./sdl_sample --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora
```

#### 必須オプション
@@ -160,8 +160,6 @@ Windows 以外の場合
- `--video-codec-type` : [ビデオコーデック指定](https://sora-doc.shiguredo.jp/SIGNALING#d47f4d)
- VP8 / VP9 / AV1 / H264 / H265 が指定可能ですが利用可能なコーデックはプラットフォームに依存します
- 未指定の場合は Sora のデフォルトである VP9 が利用されます
- `--multistream` : [マルチストリーム](https://sora-doc.shiguredo.jp/SIGNALING#808bc2) 機能の利用 (true/false)
- 未指定の場合は Sora の設定 (デフォルト: true) が設定されます
- `--video` : 映像の利用 (true/false)
- 未指定の場合は true が設定されます
- `--audio` : 音声の利用 (true/false)
8 changes: 4 additions & 4 deletions examples/sdl_sample/macos_arm64/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -10,20 +10,20 @@ set(WEBRTC_LIBRARY_DIR "" CACHE PATH "WebRTC のライブラリディレクト
set(WEBRTC_LIBRARY_NAME "webrtc" CACHE STRING "WebRTC のライブラリ名")
set(BOOST_ROOT "" CACHE PATH "Boost のルートディレクトリ")
set(SORA_DIR "" CACHE PATH "Sora のルートディレクトリ")
set(SDL2_DIR "" CACHE PATH "SDL2 のルートディレクトリ")
set(SDL3_DIR "" CACHE PATH "SDL3 のルートディレクトリ")
set(CLI11_DIR "" CACHE PATH "CLI11 のルートディレクトリ")

project(sora-sdl-sample C CXX)

list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL2_DIR})
list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL3_DIR})
list(APPEND CMAKE_MODULE_PATH ${SORA_DIR}/share/cmake)

set(Boost_USE_STATIC_LIBS ON)

find_package(Boost REQUIRED COMPONENTS json filesystem)
find_package(WebRTC REQUIRED)
find_package(Sora REQUIRED)
find_package(SDL2 REQUIRED)
find_package(SDL3 REQUIRED)
find_package(Threads REQUIRED)

add_executable(sdl_sample)
@@ -33,5 +33,5 @@ set_target_properties(sdl_sample PROPERTIES CXX_VISIBILITY_PRESET hidden)
target_sources(sdl_sample PRIVATE ../src/sdl_sample.cpp ../src/sdl_renderer.cpp)

target_include_directories(sdl_sample PRIVATE ${CLI11_DIR}/include)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL2::SDL2 SDL2::SDL2main)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL3::SDL3)
target_compile_definitions(sdl_sample PRIVATE CLI11_HAS_FILESYSTEM=0)
15 changes: 8 additions & 7 deletions examples/sdl_sample/macos_arm64/run.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
get_webrtc_info,
install_cli11,
install_cmake,
install_sdl2,
install_sdl3,
install_sora_and_deps,
install_webrtc,
mkdir_p,
@@ -96,10 +96,10 @@ def install_deps(
install_cmake(**install_cmake_args)
add_path(os.path.join(install_dir, "cmake", "CMake.app", "Contents", "bin"))

# SDL2
install_sdl2_args = {
"version": version["SDL2_VERSION"],
"version_file": os.path.join(install_dir, "sdl2.version"),
# SDL3
install_sdl3_args = {
"version": version["SDL3_VERSION"],
"version_file": os.path.join(install_dir, "sdl3.version"),
"source_dir": source_dir,
"build_dir": build_dir,
"install_dir": install_dir,
@@ -115,7 +115,7 @@ def install_deps(
f"-DCMAKE_SYSROOT={sysroot}",
],
}
install_sdl2(**install_sdl2_args)
install_sdl3(**install_sdl3_args)

# CLI11
install_cli11_args = {
@@ -166,12 +166,13 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
cmake_args.append(f"-DSORA_DIR={cmake_path(sora_info.sora_install_dir)}")
cmake_args.append(f"-DCLI11_DIR={cmake_path(os.path.join(install_dir, 'cli11'))}")
cmake_args.append(f"-DSDL2_DIR={cmake_path(os.path.join(install_dir, 'sdl2'))}")
cmake_args.append(f"-DSDL3_DIR={cmake_path(os.path.join(install_dir, 'sdl3'))}")

# クロスコンパイルの設定。
# 本来は toolchain ファイルに書く内容
87 changes: 58 additions & 29 deletions examples/sdl_sample/src/sdl_renderer.cpp
Original file line number Diff line number Diff line change
@@ -1,13 +1,42 @@
#include "sdl_renderer.h"

#include <algorithm>
#include <cmath>
#include <csignal>
#include <cstddef>
#include <cstdint>
#include <functional>
#include <memory>
#include <utility>

// WebRTC
#include <api/media_stream_interface.h>
#include <api/scoped_refptr.h>
#include <api/video/i420_buffer.h>
#include <api/video/video_frame.h>
#include <api/video/video_frame_buffer.h>
#include <api/video/video_rotation.h>
#include <api/video/video_source_interface.h>
#include <rtc_base/logging.h>
#include <rtc_base/synchronization/mutex.h>

// libyuv
#include <libyuv/convert_from.h>
#include <libyuv/video_common.h>
#include <rtc_base/logging.h>

// SDL3
#include <SDL3/SDL_error.h>
#include <SDL3/SDL_events.h>
#include <SDL3/SDL_init.h>
#include <SDL3/SDL_keycode.h>
#include <SDL3/SDL_mouse.h>
#include <SDL3/SDL_pixels.h>
#include <SDL3/SDL_rect.h>
#include <SDL3/SDL_render.h>
#include <SDL3/SDL_surface.h>
#include <SDL3/SDL_thread.h>
#include <SDL3/SDL_timer.h>
#include <SDL3/SDL_video.h>

#define STD_ASPECT 1.33
#define WIDE_ASPECT 1.78
@@ -22,15 +51,13 @@ SDLRenderer::SDLRenderer(int width, int height, bool fullscreen)
height_(height),
rows_(1),
cols_(1) {
if (SDL_Init(SDL_INIT_VIDEO) < 0) {
if (!SDL_Init(SDL_INIT_VIDEO)) {
RTC_LOG(LS_ERROR) << __FUNCTION__ << ": SDL_Init failed " << SDL_GetError();
return;
}

window_ =
SDL_CreateWindow("Sora C++ SDK - SDL Example", SDL_WINDOWPOS_CENTERED,
SDL_WINDOWPOS_CENTERED, width_, height_,
SDL_WINDOW_OPENGL | SDL_WINDOW_RESIZABLE);
window_ = SDL_CreateWindow("Sora C++ SDK - SDL Example", width_, height_,
SDL_WINDOW_OPENGL | SDL_WINDOW_RESIZABLE);
if (window_ == nullptr) {
RTC_LOG(LS_ERROR) << __FUNCTION__ << ": SDL_CreateWindow failed "
<< SDL_GetError();
@@ -44,7 +71,7 @@ SDLRenderer::SDLRenderer(int width, int height, bool fullscreen)
#if defined(__APPLE__)
// Apple Silicon Mac + macOS 11.0 だと、
// SDL_CreateRenderer をメインスレッドで呼ばないとエラーになる
renderer_ = SDL_CreateRenderer(window_, -1, SDL_RENDERER_ACCELERATED);
renderer_ = SDL_CreateRenderer(window_, NULL);
if (renderer_ == nullptr) {
RTC_LOG(LS_ERROR) << __FUNCTION__ << ": SDL_CreateRenderer failed "
<< SDL_GetError();
@@ -72,38 +99,40 @@ SDLRenderer::~SDLRenderer() {
}

bool SDLRenderer::IsFullScreen() {
return SDL_GetWindowFlags(window_) & SDL_WINDOW_FULLSCREEN_DESKTOP;
return SDL_GetWindowFlags(window_) & SDL_WINDOW_FULLSCREEN;
}

void SDLRenderer::SetFullScreen(bool fullscreen) {
SDL_SetWindowFullscreen(window_,
fullscreen ? SDL_WINDOW_FULLSCREEN_DESKTOP : 0);
SDL_ShowCursor(fullscreen ? SDL_DISABLE : SDL_ENABLE);
SDL_SetWindowFullscreen(window_, fullscreen);
if (fullscreen) {
SDL_HideCursor();
} else {
SDL_ShowCursor();
}
}

void SDLRenderer::PollEvent() {
SDL_Event e;
// 必ずメインスレッドから呼び出す
while (SDL_PollEvent(&e) > 0) {
if (e.type == SDL_WINDOWEVENT &&
e.window.event == SDL_WINDOWEVENT_RESIZED &&
while (SDL_PollEvent(&e)) {
if (e.type == SDL_EVENT_WINDOW_RESIZED &&
e.window.windowID == SDL_GetWindowID(window_)) {
webrtc::MutexLock lock(&sinks_lock_);
width_ = e.window.data1;
height_ = e.window.data2;
SetOutlines();
}
if (e.type == SDL_KEYUP) {
switch (e.key.keysym.sym) {
case SDLK_f:
if (e.type == SDL_EVENT_KEY_UP) {
switch (e.key.key) {
case SDLK_F:
SetFullScreen(!IsFullScreen());
break;
case SDLK_q:
case SDLK_Q:
std::raise(SIGTERM);
break;
}
}
if (e.type == SDL_QUIT) {
if (e.type == SDL_EVENT_QUIT) {
std::raise(SIGTERM);
}
}
@@ -123,7 +152,7 @@ int SDLRenderer::RenderThread() {
#if !defined(__APPLE__)
// Apple 以外の OpenGL あたりの実装だと、
// SDL_CreateRenderer を描画スレッドと同一のスレッドで呼ばないと何も表示されない
renderer_ = SDL_CreateRenderer(window_, -1, SDL_RENDERER_ACCELERATED);
renderer_ = SDL_CreateRenderer(window_, NULL);
if (renderer_ == nullptr) {
RTC_LOG(LS_ERROR) << __FUNCTION__ << ": SDL_CreateRenderer failed "
<< SDL_GetError();
@@ -153,18 +182,18 @@ int SDLRenderer::RenderThread() {
if (width == 0 || height == 0)
continue;

SDL_Surface* surface = SDL_CreateRGBSurfaceFrom(
sink->GetImage(), width, height, 32, width * 4, 0, 0, 0, 0);
SDL_Surface* surface =
SDL_CreateSurfaceFrom(width, height, SDL_PIXELFORMAT_ARGB8888,
sink->GetImage(), width * 4);
SDL_Texture* texture = SDL_CreateTextureFromSurface(renderer_, surface);
SDL_FreeSurface(surface);
SDL_DestroySurface(surface);

SDL_Rect image_rect = {0, 0, width, height};
SDL_Rect draw_rect = {sink->GetOffsetX(), sink->GetOffsetY(),
sink->GetWidth(), sink->GetHeight()};
SDL_FRect image_rect = {0, 0, (float)width, (float)height};
SDL_FRect draw_rect = {
(float)sink->GetOffsetX(), (float)sink->GetOffsetY(),
(float)sink->GetWidth(), (float)sink->GetHeight()};

// flip (自画像とか?)
// SDL_RenderCopyEx(renderer_, texture, &image_rect, &draw_rect, 0, nullptr, SDL_FLIP_HORIZONTAL);
SDL_RenderCopy(renderer_, texture, &image_rect, &draw_rect);
SDL_RenderTexture(renderer_, texture, &image_rect, &draw_rect);

SDL_DestroyTexture(texture);
}
14 changes: 8 additions & 6 deletions examples/sdl_sample/src/sdl_renderer.h
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
#ifndef SDL_RENDERER_H_
#define SDL_RENDERER_H_

#include <atomic>
#include <cstdint>
#include <functional>
#include <memory>
#include <string>
#include <utility>
#include <vector>

// SDL
#include <SDL2/SDL.h>

// Boost
#include <boost/asio.hpp>
// SDL3
#include <SDL3/SDL_render.h>
#include <SDL3/SDL_thread.h>
#include <SDL3/SDL_video.h>

// WebRTC
#include <api/media_stream_interface.h>
51 changes: 36 additions & 15 deletions examples/sdl_sample/src/sdl_sample.cpp
Original file line number Diff line number Diff line change
@@ -1,21 +1,46 @@
#include <csignal>
#include <cstdlib>
#include <functional>
#include <memory>
#include <optional>

// Sora
#include <sora/camera_device_capturer.h>
#include <sora/sora_client_context.h>

// CLI11
#include <CLI/CLI.hpp>
#include <string>
#include <utility>
#include <vector>

// Boost
#include <boost/asio/dispatch.hpp>
#include <boost/asio/executor_work_guard.hpp>
#include <boost/asio/io_context.hpp>
#include <boost/asio/signal_set.hpp>
#include <boost/json/parse.hpp>
#include <boost/json/value.hpp>
#include <boost/system/detail/error_code.hpp>

// WebRTC
#include <api/audio_options.h>
#include <api/media_stream_interface.h>
#include <api/rtc_error.h>
#include <api/rtp_receiver_interface.h>
#include <api/rtp_sender_interface.h>
#include <api/rtp_transceiver_interface.h>
#include <api/scoped_refptr.h>
#include <rtc_base/crypto_random.h>

#include "sdl_renderer.h"
#include <rtc_base/logging.h>

#ifdef _WIN32
#include <rtc_base/win/scoped_com_initializer.h>
#endif

// CLI11
#include <CLI/CLI.hpp>

// Sora C++ SDK
#include <sora/camera_device_capturer.h>
#include <sora/sora_client_context.h>
#include <sora/sora_signaling.h>

#include "sdl_renderer.h"

struct SDLSampleConfig {
std::string signaling_url;
std::string channel_id;
@@ -24,7 +49,6 @@ struct SDLSampleConfig {
std::string role;
std::string video_codec_type;
std::string audio_codec_type;
std::optional<bool> multistream;
int width = 640;
int height = 480;
boost::json::value metadata;
@@ -73,7 +97,6 @@ class SDLSample : public std::enable_shared_from_this<SDLSample>,
config.observer = shared_from_this();
config.signaling_urls.push_back(config_.signaling_url);
config.channel_id = config_.channel_id;
config.multistream = config_.multistream;
config.video = config_.video;
config.audio = config_.audio;
config.role = config_.role;
@@ -123,8 +146,8 @@ class SDLSample : public std::enable_shared_from_this<SDLSample>,
void OnPush(std::string text) override {}
void OnMessage(std::string label, std::string data) override {}

void OnTrack(webrtc::scoped_refptr<webrtc::RtpTransceiverInterface> transceiver)
override {
void OnTrack(webrtc::scoped_refptr<webrtc::RtpTransceiverInterface>
transceiver) override {
auto track = transceiver->receiver()->track();
if (track->kind() == webrtc::MediaStreamTrackInterface::kVideoKind) {
renderer_->AddTrack(
@@ -224,8 +247,6 @@ int main(int argc, char* argv[]) {
app.add_option("--metadata", metadata,
"Signaling metadata used in connect message")
->check(is_json);
add_optional_bool(app, "--multistream", config.multistream,
"Use multistream (default: none)");

// SDL に関するオプション
app.add_option("--width", config.width, "SDL window width");
8 changes: 4 additions & 4 deletions examples/sdl_sample/ubuntu-22.04_x86_64/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -10,20 +10,20 @@ set(WEBRTC_LIBRARY_DIR "" CACHE PATH "WebRTC のライブラリディレクト
set(WEBRTC_LIBRARY_NAME "webrtc" CACHE STRING "WebRTC のライブラリ名")
set(BOOST_ROOT "" CACHE PATH "Boost のルートディレクトリ")
set(SORA_DIR "" CACHE PATH "Sora のルートディレクトリ")
set(SDL2_DIR "" CACHE PATH "SDL2 のルートディレクトリ")
set(SDL3_DIR "" CACHE PATH "SDL3 のルートディレクトリ")
set(CLI11_DIR "" CACHE PATH "CLI11 のルートディレクトリ")

project(sora-sdl-sample C CXX)

list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL2_DIR})
list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL3_DIR})
list(APPEND CMAKE_MODULE_PATH ${SORA_DIR}/share/cmake)

set(Boost_USE_STATIC_LIBS ON)

find_package(Boost REQUIRED COMPONENTS json filesystem)
find_package(WebRTC REQUIRED)
find_package(Sora REQUIRED)
find_package(SDL2 REQUIRED)
find_package(SDL3 REQUIRED)
find_package(Threads REQUIRED)

add_executable(sdl_sample)
@@ -37,5 +37,5 @@ target_compile_options(sdl_sample
"$<$<COMPILE_LANGUAGE:CXX>:-isystem${LIBCXX_INCLUDE_DIR}>"
)
target_include_directories(sdl_sample PRIVATE ${CLI11_DIR}/include)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL2::SDL2 SDL2::SDL2main)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL3::SDL3)
target_compile_definitions(sdl_sample PRIVATE CLI11_HAS_FILESYSTEM=0)
15 changes: 8 additions & 7 deletions examples/sdl_sample/ubuntu-22.04_x86_64/run.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
install_cli11,
install_cmake,
install_llvm,
install_sdl2,
install_sdl3,
install_sora_and_deps,
install_webrtc,
mkdir_p,
@@ -123,10 +123,10 @@ def install_deps(
install_cmake(**install_cmake_args)
add_path(os.path.join(install_dir, "cmake", "bin"))

# SDL2
install_sdl2_args = {
"version": version["SDL2_VERSION"],
"version_file": os.path.join(install_dir, "sdl2.version"),
# SDL3
install_sdl3_args = {
"version": version["SDL3_VERSION"],
"version_file": os.path.join(install_dir, "sdl3.version"),
"source_dir": source_dir,
"build_dir": build_dir,
"install_dir": install_dir,
@@ -137,7 +137,7 @@ def install_deps(
f"-DCMAKE_CXX_COMPILER={os.path.join(webrtc_info.clang_dir, 'bin', 'clang++')}",
],
}
install_sdl2(**install_sdl2_args)
install_sdl3(**install_sdl3_args)

# CLI11
install_cli11_args = {
@@ -188,12 +188,13 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
cmake_args.append(f"-DSORA_DIR={cmake_path(sora_info.sora_install_dir)}")
cmake_args.append(f"-DCLI11_DIR={cmake_path(os.path.join(install_dir, 'cli11'))}")
cmake_args.append(f"-DSDL2_DIR={cmake_path(os.path.join(install_dir, 'sdl2'))}")
cmake_args.append(f"-DSDL3_DIR={cmake_path(os.path.join(install_dir, 'sdl3'))}")

# クロスコンパイルの設定。
# 本来は toolchain ファイルに書く内容
8 changes: 4 additions & 4 deletions examples/sdl_sample/ubuntu-24.04_armv8/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -10,20 +10,20 @@ set(WEBRTC_LIBRARY_DIR "" CACHE PATH "WebRTC のライブラリディレクト
set(WEBRTC_LIBRARY_NAME "webrtc" CACHE STRING "WebRTC のライブラリ名")
set(BOOST_ROOT "" CACHE PATH "Boost のルートディレクトリ")
set(SORA_DIR "" CACHE PATH "Sora のルートディレクトリ")
set(SDL2_DIR "" CACHE PATH "SDL2 のルートディレクトリ")
set(SDL3_DIR "" CACHE PATH "SDL3 のルートディレクトリ")
set(CLI11_DIR "" CACHE PATH "CLI11 のルートディレクトリ")

project(sora-sdl-sample C CXX)

list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL2_DIR})
list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL3_DIR})
list(APPEND CMAKE_MODULE_PATH ${SORA_DIR}/share/cmake)

set(Boost_USE_STATIC_LIBS ON)

find_package(Boost REQUIRED COMPONENTS json filesystem)
find_package(WebRTC REQUIRED)
find_package(Sora REQUIRED)
find_package(SDL2 REQUIRED)
find_package(SDL3 REQUIRED)
find_package(Threads REQUIRED)

add_executable(sdl_sample)
@@ -37,5 +37,5 @@ target_compile_options(sdl_sample
"$<$<COMPILE_LANGUAGE:CXX>:-isystem${LIBCXX_INCLUDE_DIR}>"
)
target_include_directories(sdl_sample PRIVATE ${CLI11_DIR}/include)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL2::SDL2 SDL2::SDL2main)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL3::SDL3)
target_compile_definitions(sdl_sample PRIVATE CLI11_HAS_FILESYSTEM=0)
15 changes: 8 additions & 7 deletions examples/sdl_sample/ubuntu-24.04_armv8/run.py
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
install_cmake,
install_llvm,
install_rootfs,
install_sdl2,
install_sdl3,
install_sora_and_deps,
install_webrtc,
mkdir_p,
@@ -137,11 +137,11 @@ def install_deps(
install_cmake(**install_cmake_args)
add_path(os.path.join(install_dir, "cmake", "bin"))

# SDL2
# SDL3
sysroot = os.path.join(install_dir, "rootfs")
install_sdl2_args = {
"version": version["SDL2_VERSION"],
"version_file": os.path.join(install_dir, "sdl2.version"),
install_sdl3_args = {
"version": version["SDL3_VERSION"],
"version_file": os.path.join(install_dir, "sdl3.version"),
"source_dir": source_dir,
"build_dir": build_dir,
"install_dir": install_dir,
@@ -162,7 +162,7 @@ def install_deps(
"-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY",
],
}
install_sdl2(**install_sdl2_args)
install_sdl3(**install_sdl3_args)

# CLI11
install_cli11_args = {
@@ -213,12 +213,13 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
cmake_args.append(f"-DSORA_DIR={cmake_path(sora_info.sora_install_dir)}")
cmake_args.append(f"-DCLI11_DIR={cmake_path(os.path.join(install_dir, 'cli11'))}")
cmake_args.append(f"-DSDL2_DIR={cmake_path(os.path.join(install_dir, 'sdl2'))}")
cmake_args.append(f"-DSDL3_DIR={cmake_path(os.path.join(install_dir, 'sdl3'))}")

# クロスコンパイルの設定。
# 本来は toolchain ファイルに書く内容
8 changes: 4 additions & 4 deletions examples/sdl_sample/ubuntu-24.04_x86_64/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -10,20 +10,20 @@ set(WEBRTC_LIBRARY_DIR "" CACHE PATH "WebRTC のライブラリディレクト
set(WEBRTC_LIBRARY_NAME "webrtc" CACHE STRING "WebRTC のライブラリ名")
set(BOOST_ROOT "" CACHE PATH "Boost のルートディレクトリ")
set(SORA_DIR "" CACHE PATH "Sora のルートディレクトリ")
set(SDL2_DIR "" CACHE PATH "SDL2 のルートディレクトリ")
set(SDL3_DIR "" CACHE PATH "SDL3 のルートディレクトリ")
set(CLI11_DIR "" CACHE PATH "CLI11 のルートディレクトリ")

project(sora-sdl-sample C CXX)

list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL2_DIR})
list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL3_DIR})
list(APPEND CMAKE_MODULE_PATH ${SORA_DIR}/share/cmake)

set(Boost_USE_STATIC_LIBS ON)

find_package(Boost REQUIRED COMPONENTS json filesystem)
find_package(WebRTC REQUIRED)
find_package(Sora REQUIRED)
find_package(SDL2 REQUIRED)
find_package(SDL3 REQUIRED)
find_package(Threads REQUIRED)

add_executable(sdl_sample)
@@ -37,5 +37,5 @@ target_compile_options(sdl_sample
"$<$<COMPILE_LANGUAGE:CXX>:-isystem${LIBCXX_INCLUDE_DIR}>"
)
target_include_directories(sdl_sample PRIVATE ${CLI11_DIR}/include)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL2::SDL2 SDL2::SDL2main)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL3::SDL3)
target_compile_definitions(sdl_sample PRIVATE CLI11_HAS_FILESYSTEM=0)
15 changes: 8 additions & 7 deletions examples/sdl_sample/ubuntu-24.04_x86_64/run.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
install_cli11,
install_cmake,
install_llvm,
install_sdl2,
install_sdl3,
install_sora_and_deps,
install_webrtc,
mkdir_p,
@@ -123,10 +123,10 @@ def install_deps(
install_cmake(**install_cmake_args)
add_path(os.path.join(install_dir, "cmake", "bin"))

# SDL2
install_sdl2_args = {
"version": version["SDL2_VERSION"],
"version_file": os.path.join(install_dir, "sdl2.version"),
# SDL3
install_sdl3_args = {
"version": version["SDL3_VERSION"],
"version_file": os.path.join(install_dir, "sdl3.version"),
"source_dir": source_dir,
"build_dir": build_dir,
"install_dir": install_dir,
@@ -137,7 +137,7 @@ def install_deps(
f"-DCMAKE_CXX_COMPILER={os.path.join(webrtc_info.clang_dir, 'bin', 'clang++')}",
],
}
install_sdl2(**install_sdl2_args)
install_sdl3(**install_sdl3_args)

# CLI11
install_cli11_args = {
@@ -188,12 +188,13 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
cmake_args.append(f"-DSORA_DIR={cmake_path(sora_info.sora_install_dir)}")
cmake_args.append(f"-DCLI11_DIR={cmake_path(os.path.join(install_dir, 'cli11'))}")
cmake_args.append(f"-DSDL2_DIR={cmake_path(os.path.join(install_dir, 'sdl2'))}")
cmake_args.append(f"-DSDL3_DIR={cmake_path(os.path.join(install_dir, 'sdl3'))}")

# クロスコンパイルの設定。
# 本来は toolchain ファイルに書く内容
8 changes: 4 additions & 4 deletions examples/sdl_sample/windows_x86_64/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -10,12 +10,12 @@ set(WEBRTC_LIBRARY_DIR "" CACHE PATH "WebRTC のライブラリディレクト
set(WEBRTC_LIBRARY_NAME "webrtc" CACHE STRING "WebRTC のライブラリ名")
set(BOOST_ROOT "" CACHE PATH "Boost のルートディレクトリ")
set(SORA_DIR "" CACHE PATH "Sora のルートディレクトリ")
set(SDL2_DIR "" CACHE PATH "SDL2 のルートディレクトリ")
set(SDL3_DIR "" CACHE PATH "SDL3 のルートディレクトリ")
set(CLI11_DIR "" CACHE PATH "CLI11 のルートディレクトリ")

project(sora-sdl-sample C CXX)

list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL2_DIR})
list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL3_DIR})
list(APPEND CMAKE_MODULE_PATH ${SORA_DIR}/share/cmake)

set(Boost_USE_STATIC_LIBS ON)
@@ -24,15 +24,15 @@ set(Boost_USE_STATIC_RUNTIME ON)
find_package(Boost REQUIRED COMPONENTS json filesystem)
find_package(WebRTC REQUIRED)
find_package(Sora REQUIRED)
find_package(SDL2 REQUIRED)
find_package(SDL3 REQUIRED)

add_executable(sdl_sample)
set_target_properties(sdl_sample PROPERTIES CXX_STANDARD 20 C_STANDARD 20)
set_target_properties(sdl_sample PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_sources(sdl_sample PRIVATE ../src/sdl_sample.cpp ../src/sdl_renderer.cpp)

target_include_directories(sdl_sample PRIVATE ${CLI11_DIR}/include)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL2::SDL2 SDL2::SDL2main)
target_link_libraries(sdl_sample PRIVATE Sora::sora SDL3::SDL3)

# 文字コードを utf-8 として扱うのと、シンボルテーブル数を増やす
target_compile_options(sdl_sample PRIVATE /utf-8 /bigobj)
15 changes: 8 additions & 7 deletions examples/sdl_sample/windows_x86_64/run.py
Original file line number Diff line number Diff line change
@@ -23,7 +23,7 @@
get_webrtc_info,
install_cli11,
install_cmake,
install_sdl2,
install_sdl3,
install_sora_and_deps,
install_webrtc,
mkdir_p,
@@ -94,18 +94,18 @@ def install_deps(

add_path(os.path.join(install_dir, "cmake", "bin"))

# SDL2
install_sdl2_args = {
"version": version["SDL2_VERSION"],
"version_file": os.path.join(install_dir, "sdl2.version"),
# SDL3
install_sdl3_args = {
"version": version["SDL3_VERSION"],
"version_file": os.path.join(install_dir, "sdl3.version"),
"source_dir": source_dir,
"build_dir": build_dir,
"install_dir": install_dir,
"debug": debug,
"platform": "windows",
"cmake_args": [],
}
install_sdl2(**install_sdl2_args)
install_sdl3(**install_sdl3_args)

# CLI11
install_cli11_args = {
@@ -156,12 +156,13 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
cmake_args.append(f"-DSORA_DIR={cmake_path(sora_info.sora_install_dir)}")
cmake_args.append(f"-DCLI11_DIR={cmake_path(os.path.join(install_dir, 'cli11'))}")
cmake_args.append(f"-DSDL2_DIR={cmake_path(os.path.join(install_dir, 'sdl2'))}")
cmake_args.append(f"-DSDL3_DIR={cmake_path(os.path.join(install_dir, 'sdl3'))}")
cmd(["cmake", os.path.join(PROJECT_DIR)] + cmake_args)
cmd(
["cmake", "--build", ".", f"-j{multiprocessing.cpu_count()}", "--config", configuration]
6 changes: 2 additions & 4 deletions examples/sumomo/README.md
Original file line number Diff line number Diff line change
@@ -132,13 +132,13 @@ _build/ubuntu-24.04_x86_64/release/sumomo/
Windows の場合

```powershell
> .\sumomo.exe --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora --multistream true --use-sdl
> .\sumomo.exe --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora --use-sdl
```

Windows 以外の場合

```shell
./sumomo --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora --multistream true --use-sdl
./sumomo --signaling-url wss://sora.example.com/signaling --role sendrecv --channel-id sora --use-sdl
```

#### 必須オプション
@@ -186,8 +186,6 @@ Windows 以外の場合
- 0 は未指定と見なされます
- `--metadata` : [メタデータ](https://sora-doc.shiguredo.jp/SIGNALING#414142)
- JSON 形式の文字列を指定してください
- `--multistream` : [マルチストリーム](https://sora-doc.shiguredo.jp/SIGNALING#808bc2) 機能の利用 (true/false)
- 未指定の場合は Sora の設定 (デフォルト: true) が設定されます
- `--spotlight` : [スポットライト](https://sora-doc.shiguredo.jp/SIGNALING#8f6c79) 機能の利用 (true/false)
- 未指定の場合は Sora の設定 (デフォルト: false) が設定されます
- `--spotlight-number` : [spotlight_number](https://sora-doc.shiguredo.jp/SPOTLIGHT#c66032)
16 changes: 11 additions & 5 deletions examples/sumomo/macos_arm64/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -10,28 +10,34 @@ set(WEBRTC_LIBRARY_DIR "" CACHE PATH "WebRTC のライブラリディレクト
set(WEBRTC_LIBRARY_NAME "webrtc" CACHE STRING "WebRTC のライブラリ名")
set(BOOST_ROOT "" CACHE PATH "Boost のルートディレクトリ")
set(SORA_DIR "" CACHE PATH "Sora のルートディレクトリ")
set(SDL2_DIR "" CACHE PATH "SDL2 のルートディレクトリ")
set(SDL3_DIR "" CACHE PATH "SDL3 のルートディレクトリ")
set(CLI11_DIR "" CACHE PATH "CLI11 のルートディレクトリ")

project(sora-sumomo C CXX)

list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL2_DIR})
list(APPEND CMAKE_PREFIX_PATH ${SORA_DIR} ${SDL3_DIR})
list(APPEND CMAKE_MODULE_PATH ${SORA_DIR}/share/cmake)

set(Boost_USE_STATIC_LIBS ON)

find_package(Boost REQUIRED COMPONENTS json filesystem)
find_package(WebRTC REQUIRED)
find_package(Sora REQUIRED)
find_package(SDL2 REQUIRED)
find_package(SDL3 REQUIRED)
find_package(Threads REQUIRED)

add_executable(sumomo)
set_target_properties(sumomo PROPERTIES CXX_STANDARD 20 C_STANDARD 20)
set_target_properties(sumomo PROPERTIES POSITION_INDEPENDENT_CODE ON)
set_target_properties(sumomo PROPERTIES CXX_VISIBILITY_PRESET hidden)
target_sources(sumomo PRIVATE ../src/sumomo.cpp ../src/sdl_renderer.cpp)
target_sources(sumomo
PRIVATE
../src/sumomo.cpp
../src/ansi_renderer.cpp
../src/base_renderer.cpp
../src/sdl_renderer.cpp
../src/sixel_renderer.cpp)

target_include_directories(sumomo PRIVATE ${CLI11_DIR}/include)
target_link_libraries(sumomo PRIVATE Sora::sora SDL2::SDL2 SDL2::SDL2main)
target_link_libraries(sumomo PRIVATE Sora::sora SDL3::SDL3)
target_compile_definitions(sumomo PRIVATE CLI11_HAS_FILESYSTEM=0)
15 changes: 8 additions & 7 deletions examples/sumomo/macos_arm64/run.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
get_webrtc_info,
install_cli11,
install_cmake,
install_sdl2,
install_sdl3,
install_sora_and_deps,
install_webrtc,
mkdir_p,
@@ -96,10 +96,10 @@ def install_deps(
install_cmake(**install_cmake_args)
add_path(os.path.join(install_dir, "cmake", "CMake.app", "Contents", "bin"))

# SDL2
install_sdl2_args = {
"version": version["SDL2_VERSION"],
"version_file": os.path.join(install_dir, "sdl2.version"),
# SDL3
install_sdl3_args = {
"version": version["SDL3_VERSION"],
"version_file": os.path.join(install_dir, "sdl3.version"),
"source_dir": source_dir,
"build_dir": build_dir,
"install_dir": install_dir,
@@ -115,7 +115,7 @@ def install_deps(
f"-DCMAKE_SYSROOT={sysroot}",
],
}
install_sdl2(**install_sdl2_args)
install_sdl3(**install_sdl3_args)

# CLI11
install_cli11_args = {
@@ -166,12 +166,13 @@ def main():

cmake_args = []
cmake_args.append(f"-DCMAKE_BUILD_TYPE={configuration}")
cmake_args.append("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON")
cmake_args.append(f"-DBOOST_ROOT={cmake_path(sora_info.boost_install_dir)}")
cmake_args.append(f"-DWEBRTC_INCLUDE_DIR={cmake_path(webrtc_info.webrtc_include_dir)}")
cmake_args.append(f"-DWEBRTC_LIBRARY_DIR={cmake_path(webrtc_info.webrtc_library_dir)}")
cmake_args.append(f"-DSORA_DIR={cmake_path(sora_info.sora_install_dir)}")
cmake_args.append(f"-DCLI11_DIR={cmake_path(os.path.join(install_dir, 'cli11'))}")
cmake_args.append(f"-DSDL2_DIR={cmake_path(os.path.join(install_dir, 'sdl2'))}")
cmake_args.append(f"-DSDL3_DIR={cmake_path(os.path.join(install_dir, 'sdl3'))}")

# クロスコンパイルの設定。
# 本来は toolchain ファイルに書く内容
105 changes: 105 additions & 0 deletions examples/sumomo/src/ansi_renderer.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
#include "ansi_renderer.h"

#include <cstdint>
#include <iostream>
#include <string>
#include <vector>

#include "base_renderer.h"

// WebRTC

AnsiRenderer::AnsiRenderer(int width, int height)
: BaseRenderer(width, height, 10) {
Start();
}

AnsiRenderer::~AnsiRenderer() {
Stop();
}

void AnsiRenderer::RenderThreadStarted() {}
void AnsiRenderer::RenderThreadFinished() {}

void AnsiRenderer::Render(uint8_t* image,
int width,
int height,
const std::vector<SinkInfo>& sink_infos) {
MoveCursorToTop();
// オリジナルサイズを表示
for (const auto& sink_info : sink_infos) {
std::cout << "\033[2K" << "(" << sink_info.offset_x << ","
<< sink_info.offset_y << ")"
<< " の元のサイズ: " << sink_info.input_width << "x"
<< sink_info.input_height << std::endl;
}
OutputAnsi(image, width, height);
}

void AnsiRenderer::ClearScreen() {
std::cout << "\033[2J\033[H" << std::flush;
}

void AnsiRenderer::MoveCursorToTop() {
std::cout << "\033[H" << std::flush;
}

int AnsiRenderer::RgbToAnsi256(uint8_t r, uint8_t g, uint8_t b) {
// 216色キューブ(6x6x6)を使用
// RGB値を0-5の範囲に変換
int r6 = (r * 5) / 255;
int g6 = (g * 5) / 255;
int b6 = (b * 5) / 255;

// ANSI 256色の216色キューブは16から始まる
return 16 + (r6 * 36) + (g6 * 6) + b6;
}

std::string AnsiRenderer::RgbToAnsi(uint8_t r, uint8_t g, uint8_t b) {
// ANSI 256色を使用してより正確な色表現
int color_code = RgbToAnsi256(r, g, b);
return "\033[48;5;" + std::to_string(color_code) + "m";
}

void AnsiRenderer::OutputAnsi(const uint8_t* rgb_data, int width, int height) {
std::string output;
output.reserve(width * height * 20); // 大体のサイズを予約

// 2x1ピクセルを1文字で表現(上半分と下半分の色を使用)
for (int y = 0; y < height; y += 2) {
output += "\033[2K"; // 行をクリア

for (int x = 0; x < width; x++) {
// 上のピクセル(y)
int upper_offset = (y * width + x) * 4;
uint8_t upper_r = rgb_data[upper_offset + 2];
uint8_t upper_g = rgb_data[upper_offset + 1];
uint8_t upper_b = rgb_data[upper_offset + 0];

// 下のピクセル(y+1)
uint8_t lower_r = upper_r, lower_g = upper_g, lower_b = upper_b;
if (y + 1 < height) {
int lower_offset = ((y + 1) * width + x) * 4;
lower_r = rgb_data[lower_offset + 2];
lower_g = rgb_data[lower_offset + 1];
lower_b = rgb_data[lower_offset + 0];
}

// 上半分の色を前景色、下半分の色を背景色として設定
int upper_color = RgbToAnsi256(upper_r, upper_g, upper_b);
int lower_color = RgbToAnsi256(lower_r, lower_g, lower_b);

// 上半分ブロック文字(▀)を使用
output += "\033[38;5;";
output += std::to_string(upper_color);
output += "m\033[48;5;";
output += std::to_string(lower_color);
output += "m▀";
}

output += "\033[0m\n"; // 色をリセットして改行
}

// 一括出力
std::cout << output << std::flush;
}
32 changes: 32 additions & 0 deletions examples/sumomo/src/ansi_renderer.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#ifndef ANSI_RENDERER_H_
#define ANSI_RENDERER_H_

#include <cstdint>
#include <string>
#include <vector>

#include "base_renderer.h"

class AnsiRenderer : public BaseRenderer {
public:
AnsiRenderer(int width, int height);
~AnsiRenderer() override;

void RenderThreadStarted() override;
void RenderThreadFinished() override;
void Render(uint8_t* image,
int width,
int height,
const std::vector<SinkInfo>& sink_infos) override;

private:
void OutputAnsi(const uint8_t* rgb_data, int width, int height);
void ClearScreen();
void MoveCursorToTop();
std::string RgbToAnsi(uint8_t r, uint8_t g, uint8_t b);

// ANSI 256色パレットへの変換用
int RgbToAnsi256(uint8_t r, uint8_t g, uint8_t b);
};

#endif
330 changes: 330 additions & 0 deletions examples/sumomo/src/base_renderer.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,330 @@
#include "base_renderer.h"

#include <algorithm>
#include <chrono>
#include <cmath>
#include <cstdint>
#include <cstring>
#include <memory>
#include <thread>
#include <utility>
#include <vector>

// WebRTC
#include <api/media_stream_interface.h>
#include <api/scoped_refptr.h>
#include <api/video/i420_buffer.h>
#include <api/video/video_frame.h>
#include <api/video/video_frame_buffer.h>
#include <api/video/video_rotation.h>
#include <api/video/video_source_interface.h>
#include <rtc_base/logging.h>
#include <rtc_base/synchronization/mutex.h>

// libyuv
#include <libyuv/convert_from.h>
#include <libyuv/planar_functions.h>
#include <libyuv/video_common.h>

static constexpr float STD_ASPECT = 1.33f; // 4:3
static constexpr float WIDE_ASPECT = 1.78f; // 16:9

BaseRenderer::BaseRenderer(int width, int height, int fps)
: running_(false),
width_(width),
height_(height),
fps_(fps),
rows_(1),
cols_(1) {}

BaseRenderer::~BaseRenderer() {
Stop();
}

void BaseRenderer::Start() {
Stop();
running_ = true;
thread_.reset(new std::thread([this]() { RenderThread(); }));
}

void BaseRenderer::Stop() {
if (thread_ != nullptr) {
running_ = false;
thread_->join();
thread_ = nullptr;
}
}

webrtc::Mutex* BaseRenderer::GetMutex() {
return &sinks_lock_;
}

void BaseRenderer::SetSize(int width, int height) {
webrtc::MutexLock lock(&sinks_lock_);
width_ = width;
height_ = height;
SetOutlines();
}

void BaseRenderer::RenderThread() {
RenderThreadStarted();

std::unique_ptr<uint8_t[]> image(new uint8_t[width_ * height_ * 4]);

while (running_) {
memset(image.get(), 0, width_ * height_ * 4);

auto frame_start = std::chrono::steady_clock::now();
std::vector<SinkInfo> sink_infos;
{
webrtc::MutexLock lock(&sinks_lock_);
for (const VideoTrackSinkVector::value_type& sinks : sinks_) {
Sink* sink = sinks.second.get();

webrtc::MutexLock frame_lock(sink->GetMutex());

if (sink->GetOutlineChanged()) {
continue;
}

int width = sink->GetFrameWidth();
int height = sink->GetFrameHeight();

if (width == 0 || height == 0) {
continue;
}

libyuv::ARGBCopy(sink->GetImage(), width * 4,
image.get() + sink->GetOffsetX() * 4 +
sink->GetOffsetY() * width_ * 4,
width_ * 4, width, height);

SinkInfo info;
info.offset_x = sink->GetOffsetX();
info.offset_y = sink->GetOffsetY();
info.input_width = sink->GetInputWidth();
info.input_height = sink->GetInputHeight();
info.frame_width = sink->GetFrameWidth();
info.frame_height = sink->GetFrameHeight();
info.width = sink->GetWidth();
info.height = sink->GetHeight();
sink_infos.push_back(info);
}
}

Render(image.get(), width_, height_, sink_infos);

// フレームレート制御
auto frame_end = std::chrono::steady_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
frame_end - frame_start)
.count();
int frame_interval = 1000 / fps_;
if (elapsed < frame_interval) {
std::this_thread::sleep_for(
std::chrono::milliseconds(frame_interval - elapsed));
}
}

RenderThreadFinished();
}

BaseRenderer::Sink::Sink(BaseRenderer* renderer,
webrtc::VideoTrackInterface* track)
: renderer_(renderer),
track_(track),
outline_offset_x_(0),
outline_offset_y_(0),
outline_width_(0),
outline_height_(0),
outline_changed_(false),
input_width_(0),
input_height_(0),
scaled_(false),
width_(0),
height_(0) {
track_->AddOrUpdateSink(this, webrtc::VideoSinkWants());
}

BaseRenderer::Sink::~Sink() {
track_->RemoveSink(this);
}

void BaseRenderer::Sink::OnFrame(const webrtc::VideoFrame& frame) {
if (outline_width_ == 0 || outline_height_ == 0)
return;
if (frame.width() == 0 || frame.height() == 0)
return;
webrtc::MutexLock lock(GetMutex());
if (outline_changed_ || frame.width() != input_width_ ||
frame.height() != input_height_) {
int width, height;
float frame_aspect = (float)frame.width() / (float)frame.height();
if (frame_aspect > outline_aspect_) {
width = outline_width_;
height = width / frame_aspect;
offset_x_ = 0;
offset_y_ = (outline_height_ - height) / 2;
} else {
height = outline_height_;
width = height * frame_aspect;
offset_x_ = (outline_width_ - width) / 2;
offset_y_ = 0;
}
if (width_ != width || height_ != height) {
width_ = width;
height_ = height;
}
input_width_ = frame.width();
input_height_ = frame.height();
scaled_ = width_ < input_width_;
if (scaled_) {
image_.reset(new uint8_t[width_ * height_ * 4]);
} else {
image_.reset(new uint8_t[input_width_ * input_height_ * 4]);
}
RTC_LOG(LS_VERBOSE) << __FUNCTION__ << ": scaled_=" << scaled_;
outline_changed_ = false;
}
webrtc::scoped_refptr<webrtc::I420BufferInterface> buffer_if;
if (scaled_) {
webrtc::scoped_refptr<webrtc::I420Buffer> buffer =
webrtc::I420Buffer::Create(width_, height_);
buffer->ScaleFrom(*frame.video_frame_buffer()->ToI420());
if (frame.rotation() != webrtc::kVideoRotation_0) {
buffer = webrtc::I420Buffer::Rotate(*buffer, frame.rotation());
}
buffer_if = buffer;
} else {
buffer_if = frame.video_frame_buffer()->ToI420();
}
libyuv::ConvertFromI420(
buffer_if->DataY(), buffer_if->StrideY(), buffer_if->DataU(),
buffer_if->StrideU(), buffer_if->DataV(), buffer_if->StrideV(),
image_.get(), (scaled_ ? width_ : input_width_) * 4, buffer_if->width(),
buffer_if->height(), libyuv::FOURCC_ARGB);
}

void BaseRenderer::Sink::SetOutlineRect(int x, int y, int width, int height) {
outline_offset_x_ = x;
outline_offset_y_ = y;
if (outline_width_ == width && outline_height_ == height) {
return;
}
webrtc::MutexLock lock(GetMutex());
offset_y_ = 0;
offset_x_ = 0;
outline_width_ = width;
outline_height_ = height;
outline_aspect_ = (float)outline_width_ / (float)outline_height_;
outline_changed_ = true;
}

webrtc::Mutex* BaseRenderer::Sink::GetMutex() {
return &frame_params_lock_;
}

bool BaseRenderer::Sink::GetOutlineChanged() {
return outline_changed_;
}

int BaseRenderer::Sink::GetOffsetX() {
return outline_offset_x_ + offset_x_;
}

int BaseRenderer::Sink::GetOffsetY() {
return outline_offset_y_ + offset_y_;
}

int BaseRenderer::Sink::GetInputWidth() {
return input_width_;
}

int BaseRenderer::Sink::GetInputHeight() {
return input_height_;
}

int BaseRenderer::Sink::GetFrameWidth() {
return scaled_ ? width_ : input_width_;
}

int BaseRenderer::Sink::GetFrameHeight() {
return scaled_ ? height_ : input_height_;
}

int BaseRenderer::Sink::GetWidth() {
return width_;
}

int BaseRenderer::Sink::GetHeight() {
return height_;
}

uint8_t* BaseRenderer::Sink::GetImage() {
return image_.get();
}

void BaseRenderer::SetOutlines() {
float window_aspect = (float)width_ / (float)height_;
bool window_is_wide = window_aspect > ((STD_ASPECT + WIDE_ASPECT) / 2.0);
float frame_aspect = window_is_wide ? WIDE_ASPECT : STD_ASPECT;
int rows = 1;
int cols = 1;
if (window_aspect >= frame_aspect) {
int times = std::floor(window_aspect / frame_aspect);
if (times < 1)
times = 1;
while (rows * cols < sinks_.size()) {
if (times < (cols / rows)) {
rows++;
} else {
cols++;
}
}
} else {
int times = std::floor(frame_aspect / window_aspect);
if (times < 1)
times = 1;
while (rows * cols < sinks_.size()) {
if (times < (rows / cols)) {
cols++;
} else {
rows++;
}
}
}
RTC_LOG(LS_VERBOSE) << __FUNCTION__ << " rows:" << rows << " cols:" << cols;
int outline_width = std::floor(width_ / cols);
int outline_height = std::floor(height_ / rows);
int sinks_count = sinks_.size();
for (int i = 0; i < sinks_count; i++) {
Sink* sink = sinks_[i].second.get();
int offset_x = outline_width * (i % cols);
int offset_y = outline_height * std::floor(i / cols);
sink->SetOutlineRect(offset_x, offset_y, outline_width, outline_height);
RTC_LOG(LS_VERBOSE) << __FUNCTION__ << " offset_x:" << offset_x
<< " offset_y:" << offset_y
<< " outline_width:" << outline_width
<< " outline_height:" << outline_height;
}
rows_ = rows;
cols_ = cols;
}

void BaseRenderer::AddTrack(webrtc::VideoTrackInterface* track) {
std::unique_ptr<Sink> sink(new Sink(this, track));
webrtc::MutexLock lock(&sinks_lock_);
sinks_.push_back(std::make_pair(track, std::move(sink)));
SetOutlines();
}

void BaseRenderer::RemoveTrack(webrtc::VideoTrackInterface* track) {
webrtc::MutexLock lock(&sinks_lock_);
sinks_.erase(
std::remove_if(sinks_.begin(), sinks_.end(),
[track](const VideoTrackSinkVector::value_type& sink) {
return sink.first == track;
}),
sinks_.end());
SetOutlines();
}
113 changes: 113 additions & 0 deletions examples/sumomo/src/base_renderer.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
#ifndef BASE_RENDERER_H_
#define BASE_RENDERER_H_

#include <atomic>
#include <cstdint>
#include <memory>
#include <thread>
#include <utility>
#include <vector>

// WebRTC
#include <api/media_stream_interface.h>
#include <api/scoped_refptr.h>
#include <api/video/video_frame.h>
#include <api/video/video_sink_interface.h>
#include <rtc_base/synchronization/mutex.h>

class BaseRenderer {
public:
BaseRenderer(int width, int height, int fps);
virtual ~BaseRenderer();

void Start();
void Stop();

void SetSize(int width, int height);
webrtc::Mutex* GetMutex();

void AddTrack(webrtc::VideoTrackInterface* track);
void RemoveTrack(webrtc::VideoTrackInterface* track);

struct SinkInfo {
int offset_x;
int offset_y;
// 入力フレームそのままのサイズ
int input_width;
int input_height;
// イメージ領域のサイズ
int frame_width;
int frame_height;
// 分割された領域のサイズ
int width;
int height;
};
virtual void RenderThreadStarted() = 0;
virtual void RenderThreadFinished() = 0;
virtual void Render(uint8_t* image,
int width,
int height,
const std::vector<SinkInfo>& sink_infos) = 0;

private:
class Sink : public webrtc::VideoSinkInterface<webrtc::VideoFrame> {
public:
Sink(BaseRenderer* renderer, webrtc::VideoTrackInterface* track);
~Sink();

void OnFrame(const webrtc::VideoFrame& frame) override;

void SetOutlineRect(int x, int y, int width, int height);

webrtc::Mutex* GetMutex();
bool GetOutlineChanged();
int GetOffsetX();
int GetOffsetY();
int GetInputWidth();
int GetInputHeight();
int GetFrameWidth();
int GetFrameHeight();
int GetWidth();
int GetHeight();
uint8_t* GetImage();

private:
BaseRenderer* renderer_;
webrtc::scoped_refptr<webrtc::VideoTrackInterface> track_;
webrtc::Mutex frame_params_lock_;
int outline_offset_x_;
int outline_offset_y_;
int outline_width_;
int outline_height_;
bool outline_changed_;
float outline_aspect_;
int input_width_;
int input_height_;
bool scaled_;
std::unique_ptr<uint8_t[]> image_;
int offset_x_;
int offset_y_;
int width_;
int height_;
};

private:
void RenderThread();
void SetOutlines();

private:
webrtc::Mutex sinks_lock_;
typedef std::vector<
std::pair<webrtc::VideoTrackInterface*, std::unique_ptr<Sink>>>
VideoTrackSinkVector;
VideoTrackSinkVector sinks_;
std::atomic<bool> running_;
std::unique_ptr<std::thread> thread_;
int width_;
int height_;
int fps_;
int rows_;
int cols_;
};

#endif
345 changes: 64 additions & 281 deletions examples/sumomo/src/sdl_renderer.cpp

Large diffs are not rendered by default.

85 changes: 13 additions & 72 deletions examples/sumomo/src/sdl_renderer.h
Original file line number Diff line number Diff line change
@@ -1,97 +1,38 @@
#ifndef SDL_RENDERER_H_
#define SDL_RENDERER_H_

#include <memory>
#include <string>
#include <cstdint>
#include <functional>
#include <vector>

// SDL
#include <SDL2/SDL.h>
#include <SDL3/SDL_render.h>
#include <SDL3/SDL_video.h>

// Boost
#include <boost/asio.hpp>
#include "base_renderer.h"

// WebRTC
#include <api/media_stream_interface.h>
#include <api/scoped_refptr.h>
#include <api/video/video_frame.h>
#include <api/video/video_sink_interface.h>
#include <rtc_base/synchronization/mutex.h>

class SDLRenderer {
class SDLRenderer : public BaseRenderer {
public:
SDLRenderer(int width, int height, bool fullscreen);
~SDLRenderer();
~SDLRenderer() override;

void SetDispatchFunction(std::function<void(std::function<void()>)> dispatch);

static int RenderThreadExec(void* data);
int RenderThread();

void SetOutlines();

void AddTrack(webrtc::VideoTrackInterface* track);
void RemoveTrack(webrtc::VideoTrackInterface* track);

protected:
class Sink : public webrtc::VideoSinkInterface<webrtc::VideoFrame> {
public:
Sink(SDLRenderer* renderer, webrtc::VideoTrackInterface* track);
~Sink();

void OnFrame(const webrtc::VideoFrame& frame) override;

void SetOutlineRect(int x, int y, int width, int height);

webrtc::Mutex* GetMutex();
bool GetOutlineChanged();
int GetOffsetX();
int GetOffsetY();
int GetFrameWidth();
int GetFrameHeight();
int GetWidth();
int GetHeight();
uint8_t* GetImage();

private:
SDLRenderer* renderer_;
webrtc::scoped_refptr<webrtc::VideoTrackInterface> track_;
webrtc::Mutex frame_params_lock_;
int outline_offset_x_;
int outline_offset_y_;
int outline_width_;
int outline_height_;
bool outline_changed_;
float outline_aspect_;
int input_width_;
int input_height_;
bool scaled_;
std::unique_ptr<uint8_t[]> image_;
int offset_x_;
int offset_y_;
int width_;
int height_;
};
void RenderThreadStarted() override;
void RenderThreadFinished() override;
void Render(uint8_t* image,
int width,
int height,
const std::vector<SinkInfo>& sink_infos) override;

private:
bool IsFullScreen();
void SetFullScreen(bool fullscreen);
void PollEvent();

webrtc::Mutex sinks_lock_;
typedef std::vector<
std::pair<webrtc::VideoTrackInterface*, std::unique_ptr<Sink>>>
VideoTrackSinkVector;
VideoTrackSinkVector sinks_;
std::atomic<bool> running_;
SDL_Thread* thread_;
SDL_Window* window_;
SDL_Renderer* renderer_;
std::function<void(std::function<void()>)> dispatch_;
int width_;
int height_;
int rows_;
int cols_;
};

#endif
212 changes: 212 additions & 0 deletions examples/sumomo/src/sixel_renderer.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
#include "sixel_renderer.h"

#include <climits>
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>

#include "base_renderer.h"

SixelRenderer::SixelRenderer(int width, int height)
: BaseRenderer(width, height, 10) {
InitializeColorLookupTable();
Start();
}

SixelRenderer::~SixelRenderer() {
Stop();
}

void SixelRenderer::RenderThreadStarted() {}
void SixelRenderer::RenderThreadFinished() {}
void SixelRenderer::Render(uint8_t* image,
int width,
int height,
const std::vector<SinkInfo>& sink_infos) {
MoveCursorToTop();
// オリジナルサイズを表示
for (const auto& sink_info : sink_infos) {
std::cout << "\033[2K" << "(" << sink_info.offset_x << ","
<< sink_info.offset_y << ")"
<< " の元のサイズ: " << sink_info.input_width << "x"
<< sink_info.input_height << std::endl;
}
OutputSixel(image, width, height);
}

void SixelRenderer::ClearScreen() {
std::cout << "\033[2J\033[H" << std::flush;
}

void SixelRenderer::MoveCursorToTop() {
std::cout << "\033[H" << std::flush;
}

void SixelRenderer::InitializeColorLookupTable() {
// 固定の216色パレット(6x6x6 RGB)を作成
palette_map_.clear();
int color_index = 0;

// 6段階のRGBの組み合わせで216色を作成
for (int r = 0; r < 6; r++) {
for (int g = 0; g < 6; g++) {
for (int b = 0; b < 6; b++) {
uint8_t red = (r * 255) / 5;
uint8_t green = (g * 255) / 5;
uint8_t blue = (b * 255) / 5;
uint32_t color = (red << 16) | (green << 8) | blue;
palette_map_[color] = color_index++;
}
}
}

// 残りの色でグレースケールを追加(216 + 40 = 256色)
for (int gray = 0; gray < 40 && color_index < 256; gray++) {
uint8_t value = (gray * 255) / 39;
uint32_t color = (value << 16) | (value << 8) | value;
// 既に存在する色は追加しない
if (palette_map_.find(color) == palette_map_.end()) {
palette_map_[color] = color_index++;
}
}

// RGB値を5ビットに減らした時のルックアップテーブルを作成(32x32x32 = 32768エントリ)
color_lookup_table_.resize(32 * 32 * 32);

for (int r = 0; r < 32; r++) {
for (int g = 0; g < 32; g++) {
for (int b = 0; b < 32; b++) {
// 5ビット値から8ビットに拡張(0-31 → 0-255)
// 例: 31 (11111) を 255 (11111111) に変換
// 左シフトで上位5ビットに配置し、右シフトで下位3ビットを埋める
uint8_t r8 = (r << 3) | (r >> 2);
uint8_t g8 = (g << 3) | (g >> 2);
uint8_t b8 = (b << 3) | (b >> 2);

// 最も近いパレット色を探す
int min_distance = INT_MAX;
int best_index = 0;

for (const auto& [palette_color, palette_index] : palette_map_) {
int pr = (palette_color >> 16) & 0xFF;
int pg = (palette_color >> 8) & 0xFF;
int pb = palette_color & 0xFF;

int dr = r8 - pr;
int dg = g8 - pg;
int db = b8 - pb;

int distance = dr * dr + dg * dg + db * db;

if (distance < min_distance) {
min_distance = distance;
best_index = palette_index;
}
}

// ルックアップテーブルに格納
color_lookup_table_[(r << 10) | (g << 5) | b] = best_index;
}
}
}
}

void SixelRenderer::OutputSixel(const uint8_t* rgb_data,
int width,
int height) {
std::string output;
output.reserve(width * height * 10); // 大体のサイズを予約

// 減色された画像データを作成(ルックアップテーブルを使用)
std::vector<uint8_t> indexed_image(width * height);
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int pixel_offset = (y * width + x) * 4;
uint8_t r = rgb_data[pixel_offset + 2];
uint8_t g = rgb_data[pixel_offset + 1];
uint8_t b = rgb_data[pixel_offset + 0];

// RGB値を5ビットに減らしてルックアップテーブルのインデックスを作成
int r5 = r >> 3;
int g5 = g >> 3;
int b5 = b >> 3;

// ルックアップテーブルから色インデックスを取得
int lookup_index = (r5 << 10) | (g5 << 5) | b5;
indexed_image[y * width + x] = color_lookup_table_[lookup_index];
}
}

// Sixelフォーマットでの出力開始
output += "\033Pq\"1;1;";
output += std::to_string(width);
output += ";";
output += std::to_string(height);

// グローバルカラーパレットを定義
for (const auto& [color, index] : palette_map_) {
int r = (color >> 16) & 0xFF;
int g = (color >> 8) & 0xFF;
int b = color & 0xFF;

// RGB値を0-100の範囲に変換
int r_percent = (r * 100) / 255;
int g_percent = (g * 100) / 255;
int b_percent = (b * 100) / 255;

output += "#";
output += std::to_string(index);
output += ";2;";
output += std::to_string(r_percent);
output += ";";
output += std::to_string(g_percent);
output += ";";
output += std::to_string(b_percent);
}

// 6行ずつ処理
for (int y = 0; y < height; y += 6) {
// 各色について6行分のデータを出力
for (const auto& [color, index] : palette_map_) {
std::vector<uint8_t> sixel_line;
bool has_pixels = false;

// この色の行データを作成
for (int x = 0; x < width; x++) {
uint8_t sixel_byte = 0;

for (int dy = 0; dy < 6 && y + dy < height; dy++) {
int pixel_index = (y + dy) * width + x;
if (indexed_image[pixel_index] == index) {
sixel_byte |= (1 << dy);
has_pixels = true;
}
}

sixel_line.push_back(sixel_byte);
}

// この色にピクセルがある場合のみ出力
if (has_pixels) {
output += "$"; // 行の最初に戻る

output += "#";
output += std::to_string(index);

for (uint8_t sixel_byte : sixel_line) {
// Sixel文字として出力(63を加える)
output += static_cast<char>(sixel_byte + 63);
}
}
}

output += "-"; // 次の6行へ
}

// Sixel終了
output += "\033\\";

// 一括出力
std::cout << output << std::flush;
}
33 changes: 33 additions & 0 deletions examples/sumomo/src/sixel_renderer.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#ifndef SIXEL_RENDERER_H_
#define SIXEL_RENDERER_H_

#include <cstdint>
#include <map>
#include <vector>

#include "base_renderer.h"

class SixelRenderer : public BaseRenderer {
public:
SixelRenderer(int width, int height);
~SixelRenderer() override;

void RenderThreadStarted() override;
void RenderThreadFinished() override;
void Render(uint8_t* image,
int width,
int height,
const std::vector<SinkInfo>& sink_infos) override;

private:
void OutputSixel(const uint8_t* rgb_data, int width, int height);
void ClearScreen();
void MoveCursorToTop();
void InitializeColorLookupTable();

// 色変換用ルックアップテーブル
std::vector<uint8_t> color_lookup_table_;
std::map<uint32_t, int> palette_map_;
};

#endif
154 changes: 121 additions & 33 deletions examples/sumomo/src/sumomo.cpp
Original file line number Diff line number Diff line change
@@ -1,25 +1,62 @@
// Sora
#include <sora/camera_device_capturer.h>
#include <sora/sora_client_context.h>
#include <sora/sora_video_codec.h>

#include <algorithm>
#include <csignal>
#include <cstdlib>
#include <fstream>
#include <functional>
#include <ios>
#include <iostream>
#include <iterator>
#include <memory>
#include <optional>
#include <ostream>
#include <regex>
#include <sstream>

// CLI11
#include <CLI/CLI.hpp>
#include <string>
#include <utility>
#include <vector>

// Boost
#include <boost/asio/dispatch.hpp>
#include <boost/asio/executor_work_guard.hpp>
#include <boost/asio/io_context.hpp>
#include <boost/asio/signal_set.hpp>
#include <boost/json/parse.hpp>
#include <boost/json/serialize.hpp>
#include <boost/json/value.hpp>
#include <boost/json/value_from.hpp>
#include <boost/system/detail/error_code.hpp>

// WebRTC
#include <api/audio_options.h>
#include <api/media_stream_interface.h>
#include <api/rtc_error.h>
#include <api/rtp_parameters.h>
#include <api/rtp_receiver_interface.h>
#include <api/rtp_sender_interface.h>
#include <api/rtp_transceiver_interface.h>
#include <api/scoped_refptr.h>
#include <api/video/video_codec_type.h>
#include <rtc_base/crypto_random.h>

#include "sdl_renderer.h"
#include <rtc_base/logging.h>

#ifdef _WIN32
#include <rtc_base/win/scoped_com_initializer.h>
#endif

// Sora C++ SDK
#include <sora/amf_context.h>
#include <sora/camera_device_capturer.h>
#include <sora/cuda_context.h>
#include <sora/sora_client_context.h>
#include <sora/sora_signaling.h>
#include <sora/sora_video_codec.h>

// CLI11
#include <CLI/CLI.hpp>

#include "ansi_renderer.h"
#include "sdl_renderer.h"
#include "sixel_renderer.h"

struct SumomoConfig {
std::string signaling_url;
std::string channel_id;
@@ -37,7 +74,6 @@ struct SumomoConfig {
boost::json::value video_h264_params;
boost::json::value video_h265_params;
boost::json::value metadata;
std::optional<bool> multistream;
std::optional<bool> spotlight;
int spotlight_number = 0;
std::optional<bool> simulcast;
@@ -54,6 +90,14 @@ struct SumomoConfig {
bool show_me = false;
bool fullscreen = false;

bool use_sixel = false;
int sixel_width = 640;
int sixel_height = 480;

bool use_ansi = false;
int ansi_width = 80;
int ansi_height = 40;

bool insecure = false;
std::string client_cert;
std::string client_key;
@@ -98,10 +142,20 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,

void Run() {
if (config_.use_sdl) {
renderer_.reset(new SDLRenderer(
sdl_renderer_.reset(new SDLRenderer(
config_.window_width, config_.window_height, config_.fullscreen));
}

if (config_.use_sixel) {
sixel_renderer_.reset(
new SixelRenderer(config_.sixel_width, config_.sixel_height));
}

if (config_.use_ansi) {
ansi_renderer_.reset(
new AnsiRenderer(config_.ansi_width, config_.ansi_height));
}

auto size = config_.GetSize();
if (config_.role != "recvonly") {
sora::CameraDeviceCapturerConfig cam_config;
@@ -125,7 +179,13 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
video_track_ = context_->peer_connection_factory()->CreateVideoTrack(
video_source, video_track_id);
if (config_.use_sdl && config_.show_me) {
renderer_->AddTrack(video_track_.get());
sdl_renderer_->AddTrack(video_track_.get());
}
if (config_.use_sixel && config_.show_me) {
sixel_renderer_->AddTrack(video_track_.get());
}
if (config_.use_ansi && config_.show_me) {
ansi_renderer_->AddTrack(video_track_.get());
}
}

@@ -138,7 +198,6 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
config.signaling_urls.push_back(config_.signaling_url);
config.channel_id = config_.channel_id;
config.role = config_.role;
config.multistream = config_.multistream;
config.client_id = config_.client_id;
config.video = config_.video;
config.audio = config_.audio;
@@ -149,7 +208,6 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
config.video_h264_params = config_.video_h264_params;
config.video_h265_params = config_.video_h265_params;
config.metadata = config_.metadata;
config.multistream = config_.multistream;
config.spotlight = config_.spotlight;
config.spotlight_number = config_.spotlight_number;
config.simulcast = config_.simulcast;
@@ -198,7 +256,7 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
conn_->Connect();

if (config_.use_sdl) {
renderer_->SetDispatchFunction([this](std::function<void()> f) {
sdl_renderer_->SetDispatchFunction([this](std::function<void()> f) {
if (ioc_->stopped())
return;
boost::asio::dispatch(ioc_->get_executor(), f);
@@ -224,7 +282,9 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
void OnDisconnect(sora::SoraSignalingErrorCode ec,
std::string message) override {
RTC_LOG(LS_INFO) << "OnDisconnect: " << message;
renderer_.reset();
sdl_renderer_.reset();
sixel_renderer_.reset();
ansi_renderer_.reset();
ioc_->stop();
}
void OnNotify(std::string text) override {
@@ -233,26 +293,40 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
void OnPush(std::string text) override {}
void OnMessage(std::string label, std::string data) override {}

void OnTrack(webrtc::scoped_refptr<webrtc::RtpTransceiverInterface> transceiver)
override {
if (renderer_ == nullptr) {
return;
}
void OnTrack(webrtc::scoped_refptr<webrtc::RtpTransceiverInterface>
transceiver) override {
auto track = transceiver->receiver()->track();
if (track->kind() == webrtc::MediaStreamTrackInterface::kVideoKind) {
renderer_->AddTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
if (sdl_renderer_) {
sdl_renderer_->AddTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
}
if (sixel_renderer_) {
sixel_renderer_->AddTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
}
if (ansi_renderer_) {
ansi_renderer_->AddTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
}
}
}
void OnRemoveTrack(
webrtc::scoped_refptr<webrtc::RtpReceiverInterface> receiver) override {
if (renderer_ == nullptr) {
return;
}
auto track = receiver->track();
if (track->kind() == webrtc::MediaStreamTrackInterface::kVideoKind) {
renderer_->RemoveTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
if (sdl_renderer_) {
sdl_renderer_->RemoveTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
}
if (sixel_renderer_) {
sixel_renderer_->RemoveTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
}
if (ansi_renderer_) {
ansi_renderer_->RemoveTrack(
static_cast<webrtc::VideoTrackInterface*>(track.get()));
}
}
}

@@ -265,7 +339,9 @@ class Sumomo : public std::enable_shared_from_this<Sumomo>,
webrtc::scoped_refptr<webrtc::VideoTrackInterface> video_track_;
std::shared_ptr<sora::SoraSignaling> conn_;
std::unique_ptr<boost::asio::io_context> ioc_;
std::unique_ptr<SDLRenderer> renderer_;
std::unique_ptr<SDLRenderer> sdl_renderer_;
std::unique_ptr<SixelRenderer> sixel_renderer_;
std::unique_ptr<AnsiRenderer> ansi_renderer_;
};

void add_optional_bool(CLI::App& app,
@@ -375,8 +451,6 @@ int main(int argc, char* argv[]) {
app.add_option("--metadata", metadata,
"Signaling metadata used in connect message")
->check(is_json);
add_optional_bool(app, "--multistream", config.multistream,
"Use multistream (default: none)");
add_optional_bool(app, "--spotlight", config.spotlight,
"Use spotlight (default: none)");
app.add_option("--spotlight-number", config.spotlight_number,
@@ -404,6 +478,19 @@ int main(int argc, char* argv[]) {
"Use fullscreen window for videos");
app.add_flag("--show-me", config.show_me, "Show self video");

// Sixel に関するオプション
app.add_flag("--use-sixel", config.use_sixel, "Show video using Sixel");
app.add_option("--sixel-width", config.sixel_width, "Sixel output width");
app.add_option("--sixel-height", config.sixel_height, "Sixel output height");

// ANSI に関するオプション
app.add_flag("--use-ansi", config.use_ansi,
"Show video using ANSI escape sequences");
app.add_option("--ansi-width", config.ansi_width,
"ANSI output width (in characters)");
app.add_option("--ansi-height", config.ansi_height,
"ANSI output height (in lines)");

// 証明書に関するオプション
app.add_flag("--insecure", config.insecure, "Allow insecure connection");
app.add_option("--client-cert", config.client_cert, "Client certificate file")
@@ -591,6 +678,7 @@ int main(int argc, char* argv[]) {
if (!audio_playout_device.empty()) {
context_config.audio_playout_device = audio_playout_device;
}
context_config.use_audio_device = false;
context_config.video_codec_factory_config.preference = std::invoke([&]() {
std::optional<sora::VideoCodecPreference> preference;
auto add_codec_preference =
Loading