feat: verify Tornado PGM cut sequence
This commit is contained in:
@@ -29,7 +29,7 @@
|
||||
- 기본 `DryRun`, Test 전용 인스턴스·채널·씬 allowlist 및 Live 이중 승인 안전 게이트
|
||||
- MSIX 패키지 매니페스트와 x64 게시 프로필
|
||||
|
||||
Oracle/MariaDB 연결 계층은 구현 및 실제 DB 스모크 검증을 완료했습니다. Tornado/K3D는 안전 기본값인 `DryRun`과 실제 어댑터 경계를 구현했습니다. 현재 장비의 Tornado2 본체에는 렌더 API가 없는 전용 진단으로 실제 `KTAPConnect → Disconnect`를 수행했고 Network Monitoring의 `HELLO / SUCCESS HELLO / BYE`를 확인했습니다. PGM 화면을 바꾸는 `5001 → 5006 → TAKE OUT` 시험은 별도의 명시적 승인 뒤에만 진행합니다. DB 설정은 [DB 운영 가이드](docs/DATABASE.md), 송출 설정과 롤백은 [Tornado/K3D 운영 가이드](docs/PLAYOUT.md), 원본 Scene/PageN 대조는 [송출 흐름 분석](docs/LEGACY_PLAYOUT_ANALYSIS.md), 전체 현황은 [마이그레이션 문서](docs/MIGRATION.md)를 참고하세요.
|
||||
Oracle/MariaDB 연결 계층은 구현 및 실제 DB 스모크 검증을 완료했습니다. Tornado/K3D는 안전 기본값인 `DryRun`과 실제 어댑터 경계를 구현했습니다. 현재 장비의 Tornado2 본체에는 렌더 API가 없는 전용 진단으로 실제 `KTAPConnect → Disconnect`를 수행했고, 별도로 승인된 고정 runner로 PGM의 `5001 → 5006 → TAKE OUT` 화면과 Network Monitoring의 `HELLO / LOAD_SCENE / SCENE_PREPARE / PLAY / STOPAL / BYE`를 확인했습니다. 이 성공은 일반 앱의 Live 이중 승인 게이트를 완화하지 않습니다. DB 설정은 [DB 운영 가이드](docs/DATABASE.md), 송출 설정·실제 검증 증거·롤백은 [Tornado/K3D 운영 가이드](docs/PLAYOUT.md), 원본 Scene/PageN 대조는 [송출 흐름 분석](docs/LEGACY_PLAYOUT_ANALYSIS.md), 전체 현황은 [마이그레이션 문서](docs/MIGRATION.md)를 참고하세요.
|
||||
|
||||
## Visual Studio 2026에서 실행
|
||||
|
||||
|
||||
@@ -56,7 +56,9 @@ WebView는 `https://app.mbn.local` 가상 호스트로 패키지 내부 파일
|
||||
- 완료: 기본 DryRun, Test의 단일 loopback 테스트 인스턴스·채널·씬 allowlist, Live 이중 승인
|
||||
- 완료: 현재 Tornado2 PGM 렌더 창에 대한 x64 K3D `KTAPConnect → Disconnect` 실제 왕복 및 Network Monitoring `[R] HELLO`/`[S] SUCCESS HELLO` 확인. 렌더 명령은 호출하지 않음
|
||||
- 완료: 네이티브/Interop 이중 SHA-256 핀, 프로세스 수명 파일 잠금, 실제 PGM listener 소유권 및 KTAP 지연 dispatch 차단
|
||||
- 확인 대기: 승인된 테스트 출력에서 `5001 → 5006 → TAKE OUT` 실제 호출
|
||||
- 완료: 승인된 실제 PGM에서 고정 해시의 `5001 → 5006 → TAKE OUT` 호출. `Connect → Prepare/Play(5001) → Prepare/Play(5006) → StopAll → Disconnect` 전 단계 성공, 세 관찰 구간 `5051/5052/5093ms`, PGM의 5001·5006 화면과 최종 검은 화면 확인
|
||||
- 완료: 같은 회차 Network Monitoring의 `HELLO/LOAD_SCENE/SCENE_PREPARE/PLAY/STOPAL/BYE` 기록과 39개 연속 캡처를 manifest SHA-256으로 검증. 로컬 증거와 재현 절차는 [Tornado/K3D 운영 가이드](PLAYOUT.md)에 기록
|
||||
- 완료: 첫 회차의 출력 전 Prepare 거부 원인이 cue 이중 resolve임을 확인하고, 상대 cue와 승인 검사용 절대 자산 경로를 분리하는 수정 및 회귀 테스트 적용. 결과 불명확 시 재시도 금지 원칙 유지
|
||||
- 확인 대기: 승인된 Test 환경에서 MSIX 컨텍스트의 실제 COM 활성화와 출력 관찰
|
||||
- 후속: `OnScenePlayed`/`OnCutOut`/`OnStopAll` callback 기반 장기 세션 Scene unload
|
||||
- 후속: 35개 scene builder의 복합 K3D mutation 및 `PageN`/`Nxt_PageN` 같은-scene 페이지 갱신 포팅
|
||||
|
||||
@@ -155,6 +155,57 @@ COM 경로에는 `KTAPConnect`와 성공 후 최대 한 번의 `Disconnect`만
|
||||
|
||||
x64 SDK의 네이티브 DLL과 정식 Interop은 위의 Registry64 단일 경로·reparse ancestry·AMD64 PE·운영자별 SHA-256 핀을 모두 통과해야 합니다. 두 파일을 쓰기/교체할 수 없게 연 핸들은 COM 활성화와 사용을 포함한 프로세스 수명 동안 유지합니다. Interop 로드 후에는 assembly/TypeLib 버전, COM import GUID와 허용 메서드 서명도 확인합니다. vendor DLL은 Git 또는 MSIX에 복사하지 않습니다.
|
||||
|
||||
### 실제 PGM 컷 시퀀스 검증
|
||||
|
||||
2026-07-10에 방송 운영자의 명시적 승인 아래 현재 Tornado2 PGM과 다음 고정 시퀀스의 실제 왕복을 완료했습니다.
|
||||
|
||||
```text
|
||||
Connect → Prepare(5001) → Play → 5초 관찰
|
||||
→ Prepare(5006) → Play → 5초 관찰
|
||||
→ StopAll → 5초 관찰 → Disconnect
|
||||
```
|
||||
|
||||
이 검증은 일반 `Test`/`Live` 설정을 완화하지 않는 별도 `--pgm-cuts-sequence` 경로입니다. 호출자가 장면 code나 관찰 시간을 바꿀 수 없고 숫자형 loopback, 정확한 PGM 창 제목, 단일 Tornado2 프로세스와 그 프로세스의 LISTEN 소유권을 각 SDK 명령과 관찰 구간 전후에 다시 검사합니다. `MBN_STOCK_PLAYOUT_*` 및 .NET startup hook/profiler 주입 환경 변수가 있으면 시작하지 않으며 자동 재연결과 자동 재생은 항상 꺼집니다. 사용한 컷은 다음 두 파일과 SHA-256으로 고정됩니다.
|
||||
|
||||
| 컷 | 승인 SHA-256 |
|
||||
|---|---|
|
||||
| `5001.t2s` | `99CE3B689A42D8C42BEB09A86FA10C2D7C1AEF4F50D324D81276C1A1E4C4D8A7` |
|
||||
| `5006.t2s` | `25CD0AE931F51E4E3B84CE3E6FD21A40DB85464F157A23CC3511D63B336D8757` |
|
||||
|
||||
재검증이 승인된 경우에는 임의의 `dotnet run` 대신 [Invoke-PgmCutsSequenceEvidence.ps1](../scripts/Invoke-PgmCutsSequenceEvidence.ps1)을 사용합니다. 먼저 x64 빌드 산출물과 두 컷의 변경이 없는지 독립적으로 검토하고, vendor/관리자가 승인한 네이티브·Interop 해시와 검토한 runner 산출물 해시를 인수로 전달합니다. `Approved*Sha256` 값은 실행 시점의 파일을 단순 계산해 곧바로 승인한 값으로 사용하지 않습니다. 실제 vendor 설치 경로와 컷 루트는 로컬 승인 기록에만 보관합니다.
|
||||
|
||||
```powershell
|
||||
$smoke = '<reviewed-x64-runner>\MBN_STOCK_WEBVIEW.PlayoutSmoke.dll'
|
||||
$cuts = '<approved-local-cut-root>'
|
||||
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File .\scripts\Invoke-PgmCutsSequenceEvidence.ps1 `
|
||||
-SmokeDll $smoke `
|
||||
-SceneRoot $cuts `
|
||||
-ApprovedNativeSha256 '<vendor-approved-native-sha256>' `
|
||||
-ApprovedInteropSha256 '<vendor-approved-interop-sha256>' `
|
||||
-ApprovedSmokeSha256 '<reviewed-smoke-sha256>' `
|
||||
-ApprovedPlayoutSha256 '<reviewed-playout-sha256>' `
|
||||
-ApprovedCoreSha256 '<reviewed-core-sha256>' `
|
||||
-ApprovedDepsSha256 '<reviewed-deps-sha256>' `
|
||||
-ApprovedRuntimeConfigSha256 '<reviewed-runtimeconfig-sha256>' `
|
||||
-ApprovedWindowsSdkSha256 '<reviewed-windows-sdk-sha256>' `
|
||||
-ApprovedWinRtSha256 '<reviewed-winrt-sha256>' `
|
||||
-ExpectedPgmWindowTitle PGM `
|
||||
-Port <verified-ktap-listener-port> `
|
||||
-IUnderstandPgmWillRenderCuts
|
||||
```
|
||||
|
||||
스크립트는 실행 전에 PGM과 Network Monitoring 창을 같은 화면에 배치해 기준 프레임 네 장을 남기고, 실행 중 약 500ms 간격으로 두 창을 함께 캡처합니다. runner와 관련 managed 산출물을 승인 해시로 확인해 읽기 잠금을 유지하고, runner 내부도 네이티브·Interop 및 컷 파일의 해시와 안전한 로컬 경로를 다시 확인합니다. 종료 후에는 세 `observation-start` 표식, 각 관찰 시간이 4,900ms 이상인지, 단계별 성공 순서, 안전 상태, runner 종료 code와 최소 프레임 수를 검증해 `manifest.json`에 각 파일·프레임의 해시와 함께 기록합니다. 창 위치와 topmost 상태는 마지막에 원래대로 복원합니다.
|
||||
|
||||
성공 회차의 terminal 결과는 `connect`, `prepare-first`, `take-in`, `next`, `take-out`, `disconnect`가 모두 `Success`였고 `completed: true`, `outcomeUnknown: false`, `renderCommandAttempted: true`, `disconnectAttempted: true`, `quarantineAttempted: false`였습니다. 실제 관찰 구간은 각각 `5051ms`, `5052ms`, `5093ms`였습니다. 캡처에서 PGM은 먼저 5001 화면, 다음으로 5006 화면을 표시했고 `StopAll` 뒤 마지막 관찰 구간에는 검은 화면으로 돌아왔습니다. 같은 캡처의 Network Monitoring에는 시간 순서대로 `HELLO`, `LOAD_SCENE`, `SCENE_PREPARE`, `PLAY`, 두 번째 `LOAD_SCENE`, `SCENE_PREPARE`, `PLAY`, `STOPAL`, `BYE` 요청/응답 기록이 남았습니다. `STOPAL`은 모니터에 표시된 KTAP 명령 표기입니다.
|
||||
|
||||
성공 증거는 이 장비의 Git 제외 경로 `artifacts/pgm-evidence/20260710_155214_success/manifest.json`에 보존했으며 SHA-256은 `2B3AA4F9B4AC4FD00613BE708D00BCDAA6687135EC06B8228F95F27482B230BB`입니다. 이 로컬 증거는 Git/MSIX에 포함되지 않습니다. 장기 보관할 때는 승인된 증거 저장소로 디렉터리 전체를 복사한 뒤 이 manifest 해시와 내부 프레임 해시를 다시 대조합니다. 해당 manifest는 runner/evidence 종료 code `0`, `terminalValidated: true`, 39개 프레임, 빈 `captureErrors`와 `validationErrors`도 기록합니다.
|
||||
|
||||
첫 회차는 `Connect: Success` 뒤 출력 전 `prepare-first: Rejected`로 중단되고 안전한 `Disconnect: Success`만 수행했습니다. 원인은 사전 검사에서 상대 cue를 절대 경로로 한 번 resolve한 뒤 그 이미 resolve된 cue를 `TornadoPlayoutEngine`에 전달해 엔진이 두 번째 resolve에서 거부한 것이었습니다. 수정 후 절대 경로는 승인 자산 검사와 파일 lease에만 사용하고, 엔진에는 고정 상대 cue인 `5001.t2s`와 `5006.t2s`를 전달합니다. 상대 cue와 검증용 절대 자산 경로가 분리되는 회귀 테스트도 추가했습니다. 이 회차는 `Play` 전에 결과가 명확히 거부되고 같은 대상의 Disconnect 성공까지 확인됐기 때문에 원인 수정 후 새 회차를 진행할 수 있었습니다. timeout, `OutcomeUnknown`, 대상 교체 또는 출력 결과가 불명확한 경우에는 자동·수동으로 반복하지 않고 quarantine 뒤 PGM 상태를 사람이 먼저 확인합니다.
|
||||
|
||||
PGM 전용 시퀀스의 `outputChannel`은 의도적으로 비워 원본 `MainForm` 연결과 같은 `GetScenePlayer()`를 사용합니다. 검증되지 않은 임의 채널을 추정해 `GetScenePlayerOnChannel()`을 호출하지 않기 위함입니다. TAKE OUT은 원본의 현재 운영 경로와 같이 `TakeOut(All)`을 `StopAll()`로 매핑합니다. 과거 `CutOut(10)`보다 현재 재생기의 모든 레이어를 정리해 최종 PGM이 검은 화면으로 돌아오는 동작과 일치합니다. 이 검증은 두 승인 컷의 load/prepare/play/stop 경로를 증명하지만, 아직 포팅되지 않은 35개 scene builder와 `PageN` 데이터 표현의 동등성을 증명하지는 않습니다.
|
||||
|
||||
## 모드와 안전 게이트
|
||||
|
||||
`Disabled`는 모든 송출 명령을 거부하는 운영 롤백 모드입니다. `DryRun`은 COM 없이 WebView 동작을 성공 결과로 모의합니다. `Test`와 `Live`만 등록된 COM을 사용할 수 있습니다.
|
||||
@@ -176,7 +227,7 @@ Test에서 로컬 프로세스/창 제목 검사를 원격 KTAP endpoint의 신
|
||||
1. 로컬 JSON의 `trustedLiveOutputEnabled`를 명시적으로 `true`로 설정합니다.
|
||||
2. 그 회차에만 프로세스 환경 변수 `MBN_STOCK_PLAYOUT_AUTHORIZE_LIVE_OUTPUT`을 정확히 `I_AUTHORIZE_LIVE_PROGRAM_OUTPUT_FOR_THIS_LAUNCH`로 설정합니다.
|
||||
|
||||
둘 중 하나라도 없으면 라이브 출력을 거부해야 합니다. 이 값은 사용자/시스템 영구 환경 변수로 저장하지 않으며 앱 종료 후 제거합니다. 현재 작업에는 이 승인이 주어지지 않았으므로 `Live` 및 실제 PGM `TAKE IN` 검증을 수행하지 않습니다.
|
||||
둘 중 하나라도 없으면 라이브 출력을 거부해야 합니다. 이 값은 사용자/시스템 영구 환경 변수로 저장하지 않으며 앱 종료 후 제거합니다. 위에서 완료한 PGM 컷 검증은 승인된 한 회차에만 사용할 수 있는 별도 고정 runner를 통한 것이며, 일반 앱의 `Live`를 자동 승인하거나 이 이중 게이트를 완화하지 않습니다.
|
||||
|
||||
## 안전한 스모크 절차
|
||||
|
||||
|
||||
846
scripts/Invoke-PgmCutsSequenceEvidence.ps1
Normal file
846
scripts/Invoke-PgmCutsSequenceEvidence.ps1
Normal file
@@ -0,0 +1,846 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$SmokeDll,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$SceneRoot,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedNativeSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedInteropSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedSmokeSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedPlayoutSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedCoreSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedDepsSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedRuntimeConfigSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedWindowsSdkSha256,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidatePattern('^[0-9A-Fa-f]{64}$')]
|
||||
[string]$ApprovedWinRtSha256,
|
||||
|
||||
[string]$ExpectedPgmWindowTitle = 'PGM',
|
||||
|
||||
[ValidateRange(1, 65535)]
|
||||
[int]$Port = 30001,
|
||||
|
||||
[string]$EvidenceDirectory,
|
||||
|
||||
[switch]$IUnderstandPgmWillRenderCuts
|
||||
)
|
||||
|
||||
Set-StrictMode -Version Latest
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
$acknowledgement =
|
||||
'--i-understand-this-will-render-current-pgm-tornado-cuts-5001-then-5006-and-take-them-out'
|
||||
$expectedCuts = @{
|
||||
'5001.t2s' = '99CE3B689A42D8C42BEB09A86FA10C2D7C1AEF4F50D324D81276C1A1E4C4D8A7'
|
||||
'5006.t2s' = '25CD0AE931F51E4E3B84CE3E6FD21A40DB85464F157A23CC3511D63B336D8757'
|
||||
}
|
||||
|
||||
if (-not $IUnderstandPgmWillRenderCuts) {
|
||||
throw 'The explicit PGM render acknowledgement switch is required.'
|
||||
}
|
||||
|
||||
$playoutOverrides = @(Get-ChildItem Env: | Where-Object {
|
||||
$_.Name -like 'MBN_STOCK_PLAYOUT_*'
|
||||
})
|
||||
if ($playoutOverrides.Count -ne 0) {
|
||||
throw 'MBN_STOCK_PLAYOUT environment overrides must be absent.'
|
||||
}
|
||||
|
||||
$runtimeInjectionOverrides = @(Get-ChildItem Env: | Where-Object {
|
||||
$_.Name -in @(
|
||||
'DOTNET_STARTUP_HOOKS',
|
||||
'DOTNET_ADDITIONAL_DEPS',
|
||||
'DOTNET_SHARED_STORE',
|
||||
'DOTNET_HOST_PATH',
|
||||
'CORECLR_ENABLE_PROFILING',
|
||||
'COR_ENABLE_PROFILING') -or
|
||||
$_.Name -like 'DOTNET_ROOT*' -or
|
||||
$_.Name -like 'CORECLR_*PROFILER*' -or
|
||||
$_.Name -like 'COR_*PROFILER*'
|
||||
})
|
||||
if ($runtimeInjectionOverrides.Count -ne 0) {
|
||||
throw '.NET startup-hook, additional-dependency, or profiler overrides must be absent.'
|
||||
}
|
||||
|
||||
$SmokeDll = [IO.Path]::GetFullPath($SmokeDll)
|
||||
$SceneRoot = [IO.Path]::GetFullPath($SceneRoot)
|
||||
if (-not (Test-Path -LiteralPath $SmokeDll -PathType Leaf)) {
|
||||
throw 'The playout smoke assembly does not exist.'
|
||||
}
|
||||
|
||||
if ([IO.Path]::GetFileName($SmokeDll) -cne 'MBN_STOCK_WEBVIEW.PlayoutSmoke.dll') {
|
||||
throw 'The expected playout smoke assembly name is required.'
|
||||
}
|
||||
|
||||
$runnerDirectory = [IO.Path]::GetDirectoryName($SmokeDll)
|
||||
$runnerFiles = [ordered]@{
|
||||
$SmokeDll = $ApprovedSmokeSha256
|
||||
(Join-Path $runnerDirectory 'MBN_STOCK_WEBVIEW.Playout.dll') = $ApprovedPlayoutSha256
|
||||
(Join-Path $runnerDirectory 'MBN_STOCK_WEBVIEW.Core.dll') = $ApprovedCoreSha256
|
||||
(Join-Path $runnerDirectory 'MBN_STOCK_WEBVIEW.PlayoutSmoke.deps.json') = $ApprovedDepsSha256
|
||||
(Join-Path $runnerDirectory 'MBN_STOCK_WEBVIEW.PlayoutSmoke.runtimeconfig.json') = $ApprovedRuntimeConfigSha256
|
||||
(Join-Path $runnerDirectory 'Microsoft.Windows.SDK.NET.dll') = $ApprovedWindowsSdkSha256
|
||||
(Join-Path $runnerDirectory 'WinRT.Runtime.dll') = $ApprovedWinRtSha256
|
||||
}
|
||||
|
||||
foreach ($entry in $expectedCuts.GetEnumerator()) {
|
||||
$path = Join-Path $SceneRoot $entry.Key
|
||||
if (-not (Test-Path -LiteralPath $path -PathType Leaf)) {
|
||||
throw "Required cut file is missing: $($entry.Key)"
|
||||
}
|
||||
|
||||
$actual = (Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash
|
||||
if ($actual -cne $entry.Value) {
|
||||
throw "Required cut file hash does not match: $($entry.Key)"
|
||||
}
|
||||
}
|
||||
|
||||
$tornado = @(Get-Process -Name 'Tornado2*' -ErrorAction SilentlyContinue)
|
||||
if ($tornado.Count -ne 1 -or $tornado[0].MainWindowTitle -cne $ExpectedPgmWindowTitle) {
|
||||
throw 'Exactly one Tornado2 process with the expected PGM title is required.'
|
||||
}
|
||||
|
||||
$listener = @(Get-NetTCPConnection -State Listen -LocalPort $Port -ErrorAction Stop)
|
||||
if ($listener.Count -ne 1 -or $listener[0].OwningProcess -ne $tornado[0].Id) {
|
||||
throw 'The exact Tornado2 process must own the requested KTAP listener.'
|
||||
}
|
||||
|
||||
if ([string]::IsNullOrWhiteSpace($EvidenceDirectory)) {
|
||||
$stamp = Get-Date -Format 'yyyyMMdd_HHmmss'
|
||||
$EvidenceDirectory = Join-Path $env:TEMP "MBN_STOCK_WEBVIEW_PGM_EVIDENCE_$stamp"
|
||||
}
|
||||
|
||||
$EvidenceDirectory = [IO.Path]::GetFullPath($EvidenceDirectory)
|
||||
if (Test-Path -LiteralPath $EvidenceDirectory) {
|
||||
throw 'The evidence directory must not already exist.'
|
||||
}
|
||||
|
||||
[void](New-Item -ItemType Directory -Path $EvidenceDirectory)
|
||||
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type @'
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Runtime.InteropServices;
|
||||
using System.Text;
|
||||
|
||||
public static class PgmEvidenceWindows
|
||||
{
|
||||
private const int GwlExStyle = -20;
|
||||
private const long WsExTopmost = 0x00000008L;
|
||||
public const int SwRestore = 9;
|
||||
public static readonly IntPtr HwndTopmost = new IntPtr(-1);
|
||||
public static readonly IntPtr HwndNotTopmost = new IntPtr(-2);
|
||||
public const uint SwpNoSize = 0x0001;
|
||||
public const uint SwpNoMove = 0x0002;
|
||||
public const uint SwpNoActivate = 0x0010;
|
||||
public const uint SwpShowWindow = 0x0040;
|
||||
|
||||
[StructLayout(LayoutKind.Sequential)]
|
||||
public struct Point { public int X; public int Y; }
|
||||
|
||||
[StructLayout(LayoutKind.Sequential)]
|
||||
public struct Rect { public int Left; public int Top; public int Right; public int Bottom; }
|
||||
|
||||
[StructLayout(LayoutKind.Sequential)]
|
||||
public struct WindowPlacement
|
||||
{
|
||||
public int Length;
|
||||
public int Flags;
|
||||
public int ShowCommand;
|
||||
public Point MinimumPosition;
|
||||
public Point MaximumPosition;
|
||||
public Rect NormalPosition;
|
||||
}
|
||||
|
||||
private delegate bool EnumWindowsCallback(IntPtr window, IntPtr parameter);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
private static extern bool EnumWindows(EnumWindowsCallback callback, IntPtr parameter);
|
||||
|
||||
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
|
||||
private static extern int GetWindowText(IntPtr window, StringBuilder text, int count);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
private static extern uint GetWindowThreadProcessId(IntPtr window, out uint processId);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
public static extern bool GetWindowPlacement(IntPtr window, ref WindowPlacement placement);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
public static extern bool GetWindowRect(IntPtr window, out Rect rectangle);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
public static extern bool SetWindowPlacement(IntPtr window, ref WindowPlacement placement);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
public static extern bool ShowWindow(IntPtr window, int command);
|
||||
|
||||
[DllImport("user32.dll")]
|
||||
public static extern bool SetWindowPos(
|
||||
IntPtr window,
|
||||
IntPtr insertAfter,
|
||||
int x,
|
||||
int y,
|
||||
int width,
|
||||
int height,
|
||||
uint flags);
|
||||
|
||||
[DllImport("user32.dll", EntryPoint = "GetWindowLong")]
|
||||
private static extern int GetWindowLong32(IntPtr window, int index);
|
||||
|
||||
[DllImport("user32.dll", EntryPoint = "GetWindowLongPtr")]
|
||||
private static extern IntPtr GetWindowLongPtr64(IntPtr window, int index);
|
||||
|
||||
public static IntPtr FindTopLevelWindow(uint processId, string exactTitle)
|
||||
{
|
||||
IntPtr match = IntPtr.Zero;
|
||||
EnumWindows((window, parameter) =>
|
||||
{
|
||||
uint owner;
|
||||
GetWindowThreadProcessId(window, out owner);
|
||||
if (owner != processId)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
var title = new StringBuilder(512);
|
||||
GetWindowText(window, title, title.Capacity);
|
||||
if (String.Equals(title.ToString(), exactTitle, StringComparison.Ordinal))
|
||||
{
|
||||
match = window;
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}, IntPtr.Zero);
|
||||
return match;
|
||||
}
|
||||
|
||||
public static bool IsTopmost(IntPtr window)
|
||||
{
|
||||
var style = IntPtr.Size == 8
|
||||
? GetWindowLongPtr64(window, GwlExStyle).ToInt64()
|
||||
: GetWindowLong32(window, GwlExStyle);
|
||||
return (style & WsExTopmost) != 0;
|
||||
}
|
||||
|
||||
public static string QuoteArgument(string argument)
|
||||
{
|
||||
if (argument == null)
|
||||
{
|
||||
throw new ArgumentNullException("argument");
|
||||
}
|
||||
|
||||
if (argument.Length > 0 &&
|
||||
argument.IndexOfAny(new[] { ' ', '\t', '\n', '\v', '"' }) < 0)
|
||||
{
|
||||
return argument;
|
||||
}
|
||||
|
||||
var commandLine = new StringBuilder();
|
||||
commandLine.Append('"');
|
||||
var backslashes = 0;
|
||||
foreach (var character in argument)
|
||||
{
|
||||
if (character == '\\')
|
||||
{
|
||||
backslashes++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (character == '"')
|
||||
{
|
||||
commandLine.Append('\\', backslashes * 2 + 1);
|
||||
commandLine.Append('"');
|
||||
backslashes = 0;
|
||||
continue;
|
||||
}
|
||||
|
||||
commandLine.Append('\\', backslashes);
|
||||
backslashes = 0;
|
||||
commandLine.Append(character);
|
||||
}
|
||||
|
||||
commandLine.Append('\\', backslashes * 2);
|
||||
commandLine.Append('"');
|
||||
return commandLine.ToString();
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
function Get-Placement([IntPtr]$Handle) {
|
||||
$placement = New-Object PgmEvidenceWindows+WindowPlacement
|
||||
$placement.Length = [Runtime.InteropServices.Marshal]::SizeOf($placement)
|
||||
if (-not [PgmEvidenceWindows]::GetWindowPlacement($Handle, [ref]$placement)) {
|
||||
throw 'Unable to read Tornado window placement.'
|
||||
}
|
||||
|
||||
return $placement
|
||||
}
|
||||
|
||||
function Open-ApprovedFileLease(
|
||||
[string]$Path,
|
||||
[string]$ApprovedSha256) {
|
||||
if (-not (Test-Path -LiteralPath $Path -PathType Leaf)) {
|
||||
throw "Required runner file is missing: $([IO.Path]::GetFileName($Path))"
|
||||
}
|
||||
|
||||
$stream = [IO.File]::Open(
|
||||
$Path,
|
||||
[IO.FileMode]::Open,
|
||||
[IO.FileAccess]::Read,
|
||||
[IO.FileShare]::Read)
|
||||
try {
|
||||
$sha = [Security.Cryptography.SHA256]::Create()
|
||||
try {
|
||||
$bytes = $sha.ComputeHash($stream)
|
||||
}
|
||||
finally {
|
||||
$sha.Dispose()
|
||||
}
|
||||
|
||||
$actual = -join ($bytes | ForEach-Object { $_.ToString('X2') })
|
||||
if ($actual -cne $ApprovedSha256.ToUpperInvariant()) {
|
||||
throw "Runner file hash does not match: $([IO.Path]::GetFileName($Path))"
|
||||
}
|
||||
|
||||
return [pscustomobject]@{
|
||||
Path = $Path
|
||||
Sha256 = $actual
|
||||
Stream = $stream
|
||||
}
|
||||
}
|
||||
catch {
|
||||
$stream.Dispose()
|
||||
throw
|
||||
}
|
||||
}
|
||||
|
||||
function Save-Frame([int]$Index, [long]$ElapsedMilliseconds) {
|
||||
$name = 'frame_{0:D4}_{1:D7}ms.png' -f $Index, $ElapsedMilliseconds
|
||||
$path = Join-Path $EvidenceDirectory $name
|
||||
$bitmap = New-Object Drawing.Bitmap `
|
||||
$script:captureWidth, `
|
||||
$script:captureHeight, `
|
||||
([Drawing.Imaging.PixelFormat]::Format24bppRgb)
|
||||
try {
|
||||
$graphics = [Drawing.Graphics]::FromImage($bitmap)
|
||||
try {
|
||||
$graphics.Clear([Drawing.Color]::Black)
|
||||
$graphics.CopyFromScreen(
|
||||
$script:networkCaptureRect.X,
|
||||
$script:networkCaptureRect.Y,
|
||||
0,
|
||||
0,
|
||||
$script:networkCaptureRect.Size)
|
||||
$graphics.CopyFromScreen(
|
||||
$script:pgmCaptureRect.X,
|
||||
$script:pgmCaptureRect.Y,
|
||||
$script:networkCaptureRect.Width + 20,
|
||||
0,
|
||||
$script:pgmCaptureRect.Size)
|
||||
}
|
||||
finally {
|
||||
$graphics.Dispose()
|
||||
}
|
||||
|
||||
$bitmap.Save($path, [Drawing.Imaging.ImageFormat]::Png)
|
||||
}
|
||||
finally {
|
||||
$bitmap.Dispose()
|
||||
}
|
||||
|
||||
return [pscustomobject]@{
|
||||
file = $name
|
||||
elapsedMilliseconds = $ElapsedMilliseconds
|
||||
sha256 = (Get-FileHash -LiteralPath $path -Algorithm SHA256).Hash
|
||||
}
|
||||
}
|
||||
|
||||
$pgmHandle = [IntPtr]$tornado[0].MainWindowHandle
|
||||
$networkTitle = -join ([char[]]@(
|
||||
0xB124, 0xD2B8, 0xC6CC, 0xD06C, 0x20,
|
||||
0xBAA8, 0xB2C8, 0xD130, 0xB9C1, 0x20, 0xCC3D))
|
||||
$networkHandle = [PgmEvidenceWindows]::FindTopLevelWindow(
|
||||
[uint32]$tornado[0].Id,
|
||||
$networkTitle)
|
||||
if ($pgmHandle -eq [IntPtr]::Zero -or $networkHandle -eq [IntPtr]::Zero) {
|
||||
throw 'Both the PGM and Network Monitoring windows must already be open.'
|
||||
}
|
||||
|
||||
$script:networkCaptureRect = New-Object Drawing.Rectangle 20, 20, 1183, 648
|
||||
$script:pgmCaptureRect = New-Object Drawing.Rectangle 1760, 70, 720, 543
|
||||
$script:captureWidth =
|
||||
$script:networkCaptureRect.Width + 20 + $script:pgmCaptureRect.Width
|
||||
$script:captureHeight = [Math]::Max(
|
||||
$script:networkCaptureRect.Height,
|
||||
$script:pgmCaptureRect.Height)
|
||||
$primaryBounds = [Windows.Forms.Screen]::PrimaryScreen.Bounds
|
||||
if (-not $primaryBounds.Contains($script:networkCaptureRect) -or
|
||||
-not $primaryBounds.Contains($script:pgmCaptureRect)) {
|
||||
throw 'The primary display cannot contain both evidence windows.'
|
||||
}
|
||||
|
||||
function Assert-WindowRectangle(
|
||||
[IntPtr]$Handle,
|
||||
[Drawing.Rectangle]$Expected) {
|
||||
$actual = New-Object PgmEvidenceWindows+Rect
|
||||
if (-not [PgmEvidenceWindows]::GetWindowRect($Handle, [ref]$actual) -or
|
||||
$actual.Left -ne $Expected.Left -or
|
||||
$actual.Top -ne $Expected.Top -or
|
||||
($actual.Right - $actual.Left) -ne $Expected.Width -or
|
||||
($actual.Bottom - $actual.Top) -ne $Expected.Height) {
|
||||
throw 'A Tornado evidence window was not placed at the verified capture rectangle.'
|
||||
}
|
||||
}
|
||||
|
||||
$pgmPlacement = Get-Placement $pgmHandle
|
||||
$networkPlacement = Get-Placement $networkHandle
|
||||
$pgmWasTopmost = [PgmEvidenceWindows]::IsTopmost($pgmHandle)
|
||||
$networkWasTopmost = [PgmEvidenceWindows]::IsTopmost($networkHandle)
|
||||
$dotnet = 'C:\Program Files\dotnet\dotnet.exe'
|
||||
if (-not (Test-Path -LiteralPath $dotnet -PathType Leaf)) {
|
||||
throw 'The trusted 64-bit dotnet host was not found.'
|
||||
}
|
||||
|
||||
$dotnetSignature = Get-AuthenticodeSignature -LiteralPath $dotnet
|
||||
if ($dotnetSignature.Status -ne [System.Management.Automation.SignatureStatus]::Valid -or
|
||||
$null -eq $dotnetSignature.SignerCertificate -or
|
||||
$dotnetSignature.SignerCertificate.Subject -notmatch
|
||||
'(^|,\s*)O=Microsoft Corporation(,|$)') {
|
||||
throw 'The dotnet host signature is not trusted.'
|
||||
}
|
||||
|
||||
$runnerLeases = [Collections.Generic.List[object]]::new()
|
||||
try {
|
||||
foreach ($runnerFile in $runnerFiles.GetEnumerator()) {
|
||||
$runnerLeases.Add((Open-ApprovedFileLease `
|
||||
([string]$runnerFile.Key) `
|
||||
([string]$runnerFile.Value)))
|
||||
}
|
||||
}
|
||||
catch {
|
||||
foreach ($lease in $runnerLeases) {
|
||||
$lease.Stream.Dispose()
|
||||
}
|
||||
|
||||
throw
|
||||
}
|
||||
|
||||
$frames = [Collections.Generic.List[object]]::new()
|
||||
$output = [Collections.Generic.List[object]]::new()
|
||||
$captureErrors = [Collections.Generic.List[string]]::new()
|
||||
$restorationErrors = [Collections.Generic.List[string]]::new()
|
||||
$process = $null
|
||||
$exitCode = $null
|
||||
$evidenceExitCode = $null
|
||||
$result = $null
|
||||
$stopwatch = [Diagnostics.Stopwatch]::StartNew()
|
||||
|
||||
try {
|
||||
[void][PgmEvidenceWindows]::ShowWindow($networkHandle, [PgmEvidenceWindows]::SwRestore)
|
||||
[void][PgmEvidenceWindows]::ShowWindow($pgmHandle, [PgmEvidenceWindows]::SwRestore)
|
||||
if (-not [PgmEvidenceWindows]::SetWindowPos(
|
||||
$networkHandle,
|
||||
[PgmEvidenceWindows]::HwndTopmost,
|
||||
$script:networkCaptureRect.X,
|
||||
$script:networkCaptureRect.Y,
|
||||
$script:networkCaptureRect.Width,
|
||||
$script:networkCaptureRect.Height,
|
||||
[PgmEvidenceWindows]::SwpNoActivate -bor
|
||||
[PgmEvidenceWindows]::SwpShowWindow)) {
|
||||
throw 'Unable to place the Network Monitoring window for evidence capture.'
|
||||
}
|
||||
|
||||
if (-not [PgmEvidenceWindows]::SetWindowPos(
|
||||
$pgmHandle,
|
||||
[PgmEvidenceWindows]::HwndTopmost,
|
||||
$script:pgmCaptureRect.X,
|
||||
$script:pgmCaptureRect.Y,
|
||||
$script:pgmCaptureRect.Width,
|
||||
$script:pgmCaptureRect.Height,
|
||||
[PgmEvidenceWindows]::SwpNoActivate -bor
|
||||
[PgmEvidenceWindows]::SwpShowWindow)) {
|
||||
throw 'Unable to place the PGM window for evidence capture.'
|
||||
}
|
||||
|
||||
Assert-WindowRectangle $networkHandle $script:networkCaptureRect
|
||||
Assert-WindowRectangle $pgmHandle $script:pgmCaptureRect
|
||||
Start-Sleep -Milliseconds 750
|
||||
Assert-WindowRectangle $networkHandle $script:networkCaptureRect
|
||||
Assert-WindowRectangle $pgmHandle $script:pgmCaptureRect
|
||||
|
||||
for ($index = 0; $index -lt 4; $index++) {
|
||||
$frames.Add((Save-Frame $index $stopwatch.ElapsedMilliseconds))
|
||||
Start-Sleep -Milliseconds 500
|
||||
}
|
||||
|
||||
$start = [Diagnostics.ProcessStartInfo]::new()
|
||||
$start.FileName = $dotnet
|
||||
$start.UseShellExecute = $false
|
||||
$start.CreateNoWindow = $true
|
||||
$start.RedirectStandardOutput = $true
|
||||
$start.RedirectStandardError = $true
|
||||
$runnerArguments = @(
|
||||
$SmokeDll,
|
||||
'--pgm-cuts-sequence',
|
||||
$acknowledgement,
|
||||
'--host',
|
||||
'127.0.0.1',
|
||||
'--port',
|
||||
$Port.ToString([Globalization.CultureInfo]::InvariantCulture),
|
||||
'--expected-pgm-window-title',
|
||||
$ExpectedPgmWindowTitle,
|
||||
'--scene-root',
|
||||
$SceneRoot,
|
||||
'--observe-ms',
|
||||
'5000')
|
||||
$start.Arguments = ($runnerArguments | ForEach-Object {
|
||||
[PgmEvidenceWindows]::QuoteArgument([string]$_)
|
||||
}) -join ' '
|
||||
|
||||
$start.EnvironmentVariables['MBN_STOCK_K3D_NATIVE_SHA256'] =
|
||||
$ApprovedNativeSha256.ToUpperInvariant()
|
||||
$start.EnvironmentVariables['MBN_STOCK_K3D_INTEROP_SHA256'] =
|
||||
$ApprovedInteropSha256.ToUpperInvariant()
|
||||
|
||||
$process = [Diagnostics.Process]::new()
|
||||
$process.StartInfo = $start
|
||||
if (-not $process.Start()) {
|
||||
throw 'Unable to launch the PGM sequence runner.'
|
||||
}
|
||||
|
||||
$processStartElapsed = $stopwatch.ElapsedMilliseconds
|
||||
$frameIndex = 4
|
||||
$captureEnabled = $true
|
||||
try {
|
||||
$stdoutRead = $process.StandardOutput.ReadLineAsync()
|
||||
$stderrRead = $process.StandardError.ReadLineAsync()
|
||||
$nextFrameAt = $stopwatch.ElapsedMilliseconds
|
||||
$stdoutEnded = $false
|
||||
$stderrEnded = $false
|
||||
|
||||
while (-not $process.HasExited -or -not $stdoutEnded -or -not $stderrEnded) {
|
||||
if (-not $stdoutEnded -and $stdoutRead.IsCompleted) {
|
||||
try {
|
||||
$line = $stdoutRead.GetAwaiter().GetResult()
|
||||
}
|
||||
catch {
|
||||
$captureErrors.Add('stdout-read-failed')
|
||||
$stdoutEnded = $true
|
||||
$line = $null
|
||||
}
|
||||
|
||||
if ($null -eq $line) {
|
||||
$stdoutEnded = $true
|
||||
}
|
||||
else {
|
||||
$output.Add([pscustomobject]@{
|
||||
stream = 'stdout'
|
||||
elapsedMilliseconds = $stopwatch.ElapsedMilliseconds
|
||||
line = $line
|
||||
})
|
||||
$stdoutRead = $process.StandardOutput.ReadLineAsync()
|
||||
}
|
||||
}
|
||||
|
||||
if (-not $stderrEnded -and $stderrRead.IsCompleted) {
|
||||
try {
|
||||
$line = $stderrRead.GetAwaiter().GetResult()
|
||||
}
|
||||
catch {
|
||||
$captureErrors.Add('stderr-read-failed')
|
||||
$stderrEnded = $true
|
||||
$line = $null
|
||||
}
|
||||
|
||||
if ($null -eq $line) {
|
||||
$stderrEnded = $true
|
||||
}
|
||||
else {
|
||||
$output.Add([pscustomobject]@{
|
||||
stream = 'stderr'
|
||||
elapsedMilliseconds = $stopwatch.ElapsedMilliseconds
|
||||
line = $line
|
||||
})
|
||||
$stderrRead = $process.StandardError.ReadLineAsync()
|
||||
}
|
||||
}
|
||||
|
||||
if ($captureEnabled -and $stopwatch.ElapsedMilliseconds -ge $nextFrameAt) {
|
||||
try {
|
||||
$frames.Add((Save-Frame $frameIndex $stopwatch.ElapsedMilliseconds))
|
||||
$frameIndex++
|
||||
}
|
||||
catch {
|
||||
$captureErrors.Add('frame-capture-failed')
|
||||
$captureEnabled = $false
|
||||
}
|
||||
|
||||
$nextFrameAt += 500
|
||||
}
|
||||
|
||||
Start-Sleep -Milliseconds 20
|
||||
}
|
||||
}
|
||||
catch {
|
||||
$captureErrors.Add('supervision-loop-failed')
|
||||
while (-not $process.HasExited) {
|
||||
Start-Sleep -Milliseconds 100
|
||||
}
|
||||
}
|
||||
|
||||
$process.WaitForExit()
|
||||
$exitCode = $process.ExitCode
|
||||
Start-Sleep -Milliseconds 500
|
||||
if ($captureEnabled) {
|
||||
try {
|
||||
$frames.Add((Save-Frame $frameIndex $stopwatch.ElapsedMilliseconds))
|
||||
}
|
||||
catch {
|
||||
$captureErrors.Add('final-frame-capture-failed')
|
||||
}
|
||||
}
|
||||
|
||||
$validationErrors = [Collections.Generic.List[string]]::new()
|
||||
$stdoutEntries = @($output | Where-Object { $_.stream -eq 'stdout' })
|
||||
$markers = [Collections.Generic.List[object]]::new()
|
||||
$lastMarkerIndex = -1
|
||||
for ($index = 0; $index -lt $stdoutEntries.Count; $index++) {
|
||||
$entry = $stdoutEntries[$index]
|
||||
if ($entry.line -notmatch '"event"\s*:\s*"observation-start"') {
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
$marker = $entry.line | ConvertFrom-Json -ErrorAction Stop
|
||||
$markers.Add([pscustomobject]@{
|
||||
stage = [string]$marker.stage
|
||||
elapsedMilliseconds = [long]$entry.elapsedMilliseconds
|
||||
})
|
||||
$lastMarkerIndex = $index
|
||||
}
|
||||
catch {
|
||||
$validationErrors.Add('observation-marker-json-invalid')
|
||||
}
|
||||
}
|
||||
|
||||
$markerStages = @($markers | ForEach-Object { $_.stage })
|
||||
$expectedMarkerStages = @('first', 'next', 'take-out')
|
||||
if ($markerStages.Count -ne $expectedMarkerStages.Count -or
|
||||
($markerStages -join '|') -cne ($expectedMarkerStages -join '|')) {
|
||||
$validationErrors.Add('observation-marker-order-invalid')
|
||||
}
|
||||
|
||||
$terminalEntries = if ($lastMarkerIndex + 1 -lt $stdoutEntries.Count) {
|
||||
@($stdoutEntries[($lastMarkerIndex + 1)..($stdoutEntries.Count - 1)])
|
||||
}
|
||||
else {
|
||||
@()
|
||||
}
|
||||
$terminal = $null
|
||||
if ($terminalEntries.Count -ne 0) {
|
||||
try {
|
||||
$terminal = ($terminalEntries.line -join [Environment]::NewLine) |
|
||||
ConvertFrom-Json -ErrorAction Stop
|
||||
}
|
||||
catch {
|
||||
$validationErrors.Add('terminal-report-json-invalid')
|
||||
}
|
||||
}
|
||||
|
||||
$observationIntervals = @()
|
||||
if ($markers.Count -eq 3 -and $terminalEntries.Count -ne 0) {
|
||||
$observationIntervals = @(
|
||||
([long]$markers[1].elapsedMilliseconds -
|
||||
[long]$markers[0].elapsedMilliseconds),
|
||||
([long]$markers[2].elapsedMilliseconds -
|
||||
[long]$markers[1].elapsedMilliseconds),
|
||||
([long]$terminalEntries[0].elapsedMilliseconds -
|
||||
[long]$markers[2].elapsedMilliseconds)
|
||||
)
|
||||
if (@($observationIntervals | Where-Object { $_ -lt 4900 }).Count -ne 0) {
|
||||
$validationErrors.Add('observation-duration-too-short')
|
||||
}
|
||||
}
|
||||
else {
|
||||
$validationErrors.Add('observation-duration-unavailable')
|
||||
}
|
||||
|
||||
if ($null -eq $terminal -or $terminal.command -ne 'pgm-cuts-sequence') {
|
||||
$validationErrors.Add('terminal-report-missing-or-invalid')
|
||||
}
|
||||
else {
|
||||
$actualCommandStages = @($terminal.commands | ForEach-Object {
|
||||
'{0}:{1}' -f $_.stage, $_.code
|
||||
})
|
||||
$expectedCommandStages = @(
|
||||
'connect:Success',
|
||||
'prepare-first:Success',
|
||||
'take-in:Success',
|
||||
'next:Success',
|
||||
'take-out:Success',
|
||||
'disconnect:Success')
|
||||
if ($actualCommandStages.Count -ne $expectedCommandStages.Count -or
|
||||
($actualCommandStages -join '|') -cne ($expectedCommandStages -join '|')) {
|
||||
$validationErrors.Add('terminal-command-order-invalid')
|
||||
}
|
||||
|
||||
if ($terminal.completed -ne $true -or
|
||||
$terminal.outcomeUnknown -ne $false -or
|
||||
$terminal.renderCommandAttempted -ne $true -or
|
||||
$terminal.disconnectAttempted -ne $true -or
|
||||
$terminal.quarantineAttempted -ne $false) {
|
||||
$validationErrors.Add('terminal-safety-state-invalid')
|
||||
}
|
||||
}
|
||||
|
||||
if ($exitCode -ne 0) {
|
||||
$validationErrors.Add('runner-exit-code-nonzero')
|
||||
}
|
||||
|
||||
if ($frames.Count -lt 30) {
|
||||
$validationErrors.Add('insufficient-frame-count')
|
||||
}
|
||||
|
||||
if ($captureErrors.Count -ne 0) {
|
||||
$validationErrors.Add('evidence-capture-incomplete')
|
||||
}
|
||||
|
||||
$terminalValidated = $validationErrors.Count -eq 0
|
||||
$evidenceExitCode = if ($exitCode -ne 0) { $exitCode } elseif (
|
||||
-not $terminalValidated) { 6 } else { 0 }
|
||||
|
||||
$manifest = [ordered]@{
|
||||
schemaVersion = 1
|
||||
createdAtUtc = [DateTimeOffset]::UtcNow.ToString('O')
|
||||
processStartElapsedMilliseconds = $processStartElapsed
|
||||
runnerExitCode = $exitCode
|
||||
evidenceExitCode = $evidenceExitCode
|
||||
terminalValidated = $terminalValidated
|
||||
runnerFilesSha256 = @($runnerLeases | ForEach-Object {
|
||||
[ordered]@{
|
||||
file = [IO.Path]::GetFileName($_.Path)
|
||||
sha256 = $_.Sha256
|
||||
}
|
||||
})
|
||||
cutSha256 = $expectedCuts
|
||||
frames = $frames
|
||||
captureErrors = $captureErrors
|
||||
validationErrors = $validationErrors
|
||||
observationIntervalsMilliseconds = $observationIntervals
|
||||
runnerOutput = $output
|
||||
}
|
||||
$manifestPath = Join-Path $EvidenceDirectory 'manifest.json'
|
||||
[IO.File]::WriteAllText(
|
||||
$manifestPath,
|
||||
($manifest | ConvertTo-Json -Depth 8),
|
||||
[Text.UTF8Encoding]::new($false))
|
||||
$manifestHash = (Get-FileHash -LiteralPath $manifestPath -Algorithm SHA256).Hash
|
||||
|
||||
$result = [pscustomobject]@{
|
||||
EvidenceDirectory = $EvidenceDirectory
|
||||
RunnerExitCode = $exitCode
|
||||
TerminalValidated = $terminalValidated
|
||||
FrameCount = $frames.Count
|
||||
ManifestSha256 = $manifestHash
|
||||
}
|
||||
}
|
||||
finally {
|
||||
if ($null -ne $process) {
|
||||
$process.Dispose()
|
||||
}
|
||||
|
||||
foreach ($lease in $runnerLeases) {
|
||||
$lease.Stream.Dispose()
|
||||
}
|
||||
|
||||
if (-not [PgmEvidenceWindows]::SetWindowPos(
|
||||
$networkHandle,
|
||||
$(if ($networkWasTopmost) {
|
||||
[PgmEvidenceWindows]::HwndTopmost
|
||||
} else {
|
||||
[PgmEvidenceWindows]::HwndNotTopmost
|
||||
}),
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
[PgmEvidenceWindows]::SwpNoActivate -bor
|
||||
[PgmEvidenceWindows]::SwpNoMove -bor
|
||||
[PgmEvidenceWindows]::SwpNoSize)) {
|
||||
$restorationErrors.Add('network-topmost-restore-failed')
|
||||
}
|
||||
|
||||
if (-not [PgmEvidenceWindows]::SetWindowPos(
|
||||
$pgmHandle,
|
||||
$(if ($pgmWasTopmost) {
|
||||
[PgmEvidenceWindows]::HwndTopmost
|
||||
} else {
|
||||
[PgmEvidenceWindows]::HwndNotTopmost
|
||||
}),
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
[PgmEvidenceWindows]::SwpNoActivate -bor
|
||||
[PgmEvidenceWindows]::SwpNoMove -bor
|
||||
[PgmEvidenceWindows]::SwpNoSize)) {
|
||||
$restorationErrors.Add('pgm-topmost-restore-failed')
|
||||
}
|
||||
|
||||
if (-not [PgmEvidenceWindows]::SetWindowPlacement(
|
||||
$networkHandle,
|
||||
[ref]$networkPlacement)) {
|
||||
$restorationErrors.Add('network-placement-restore-failed')
|
||||
}
|
||||
|
||||
if (-not [PgmEvidenceWindows]::SetWindowPlacement(
|
||||
$pgmHandle,
|
||||
[ref]$pgmPlacement)) {
|
||||
$restorationErrors.Add('pgm-placement-restore-failed')
|
||||
}
|
||||
}
|
||||
|
||||
if ($null -ne $result) {
|
||||
$result | Add-Member -NotePropertyName RestorationSucceeded `
|
||||
-NotePropertyValue ($restorationErrors.Count -eq 0)
|
||||
}
|
||||
|
||||
if ($restorationErrors.Count -ne 0) {
|
||||
Write-Warning ($restorationErrors -join ',')
|
||||
}
|
||||
|
||||
$result
|
||||
if ($evidenceExitCode -ne 0) {
|
||||
exit $evidenceExitCode
|
||||
}
|
||||
1065
src/MBN_STOCK_WEBVIEW.Playout/Diagnostics/PgmCutsSequence.cs
Normal file
1065
src/MBN_STOCK_WEBVIEW.Playout/Diagnostics/PgmCutsSequence.cs
Normal file
File diff suppressed because it is too large
Load Diff
@@ -29,6 +29,19 @@ internal interface IK3dSession : IDisposable
|
||||
void Abandon();
|
||||
}
|
||||
|
||||
internal interface IK3dGuardedConnectSession : IK3dSession
|
||||
{
|
||||
void ConnectGuarded(
|
||||
ValidatedPlayoutOptions options,
|
||||
Func<bool> finalSafetyCheck,
|
||||
IK3dSdkDispatchLatch dispatchLatch);
|
||||
}
|
||||
|
||||
internal interface IK3dSdkDispatchLatch
|
||||
{
|
||||
IDisposable? TryClaim();
|
||||
}
|
||||
|
||||
internal interface IK3dSessionFactory
|
||||
{
|
||||
IK3dSession Create();
|
||||
@@ -102,7 +115,7 @@ internal sealed class RuntimeComObjectReleaser : ILateBoundComObjectReleaser
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class DynamicK3dSession : IK3dSession
|
||||
internal sealed class DynamicK3dSession : IK3dGuardedConnectSession
|
||||
{
|
||||
private readonly ILateBoundComActivator _activator;
|
||||
private readonly ILateBoundComObjectReleaser _releaser;
|
||||
@@ -164,6 +177,27 @@ internal sealed class DynamicK3dSession : IK3dSession
|
||||
}
|
||||
|
||||
public void Connect(ValidatedPlayoutOptions options)
|
||||
=> ConnectCore(options, finalSafetyCheck: null, disconnectOnFailure: true);
|
||||
|
||||
public void ConnectGuarded(
|
||||
ValidatedPlayoutOptions options,
|
||||
Func<bool> finalSafetyCheck,
|
||||
IK3dSdkDispatchLatch dispatchLatch)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(finalSafetyCheck);
|
||||
ArgumentNullException.ThrowIfNull(dispatchLatch);
|
||||
ConnectCore(
|
||||
options,
|
||||
finalSafetyCheck,
|
||||
disconnectOnFailure: false,
|
||||
dispatchLatch: dispatchLatch);
|
||||
}
|
||||
|
||||
private void ConnectCore(
|
||||
ValidatedPlayoutOptions options,
|
||||
Func<bool>? finalSafetyCheck,
|
||||
bool disconnectOnFailure,
|
||||
IK3dSdkDispatchLatch? dispatchLatch = null)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
EnsureThread();
|
||||
@@ -178,22 +212,37 @@ internal sealed class DynamicK3dSession : IK3dSession
|
||||
{
|
||||
_eventHandler = _activator.Create(K3dComConstants.KaEventHandlerClassGuid);
|
||||
_engine = _activator.Create(K3dComConstants.KaEngineClassGuid);
|
||||
if (Interlocked.CompareExchange(ref _ktapDispatchGate, 1, 0) != 0)
|
||||
if (finalSafetyCheck is not null && !finalSafetyCheck())
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
"The K3D connection was abandoned before KTAP dispatch.");
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
object? result;
|
||||
using (var dispatchClaim = dispatchLatch?.TryClaim())
|
||||
{
|
||||
if (dispatchLatch is not null && dispatchClaim is null)
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
if (Interlocked.CompareExchange(ref _ktapDispatchGate, 1, 0) != 0)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
"The K3D connection was abandoned before KTAP dispatch.");
|
||||
}
|
||||
|
||||
ktapConnectAttempted = true;
|
||||
SetKtapConnectState(PlayoutKtapConnectState.Attempted);
|
||||
result = Invoke(
|
||||
_engine,
|
||||
"KTAPConnect",
|
||||
options.TcpMode,
|
||||
options.Host,
|
||||
options.Port,
|
||||
options.ClientPort,
|
||||
_eventHandler);
|
||||
}
|
||||
|
||||
ktapConnectAttempted = true;
|
||||
SetKtapConnectState(PlayoutKtapConnectState.Attempted);
|
||||
var result = Invoke(
|
||||
_engine,
|
||||
"KTAPConnect",
|
||||
options.TcpMode,
|
||||
options.Host,
|
||||
options.Port,
|
||||
options.ClientPort,
|
||||
_eventHandler);
|
||||
var hresult = result is null
|
||||
? 0
|
||||
: Convert.ToInt32(result, CultureInfo.InvariantCulture);
|
||||
@@ -206,9 +255,23 @@ internal sealed class DynamicK3dSession : IK3dSession
|
||||
SetKtapConnectState(PlayoutKtapConnectState.AcceptedUnconfirmed);
|
||||
|
||||
var outputChannel = options.OutputChannel;
|
||||
_player = outputChannel is { } channel
|
||||
? InvokeRequired(_engine, "GetScenePlayerOnChannel", channel)
|
||||
: InvokeRequired(_engine, "GetScenePlayer");
|
||||
if (finalSafetyCheck is not null && !finalSafetyCheck())
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
using (var dispatchClaim = dispatchLatch?.TryClaim())
|
||||
{
|
||||
if (dispatchLatch is not null && dispatchClaim is null)
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
_player = outputChannel is { } channel
|
||||
? InvokeRequired(_engine, "GetScenePlayerOnChannel", channel)
|
||||
: InvokeRequired(_engine, "GetScenePlayer");
|
||||
}
|
||||
|
||||
_outputChannel = outputChannel;
|
||||
}
|
||||
catch (Exception exception)
|
||||
@@ -219,7 +282,7 @@ internal sealed class DynamicK3dSession : IK3dSession
|
||||
}
|
||||
|
||||
var originalFailure = ExceptionDispatchInfo.Capture(exception);
|
||||
if (ktapConnectAttempted && _engine is not null)
|
||||
if (disconnectOnFailure && ktapConnectAttempted && _engine is not null)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -553,3 +616,5 @@ internal sealed class DynamicK3dSession : IK3dSession
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class K3dConnectSafetyException : Exception;
|
||||
|
||||
@@ -14,7 +14,8 @@ internal interface IStaDispatcher : IAsyncDisposable
|
||||
TimeSpan timeout,
|
||||
CancellationToken cancellationToken,
|
||||
Action<T>? abandonedResultCleanup = null,
|
||||
Action? abandonedFailureCleanup = null);
|
||||
Action? abandonedFailureCleanup = null,
|
||||
Action? preventFutureDispatch = null);
|
||||
}
|
||||
|
||||
internal interface IStaDispatcherFactory
|
||||
@@ -75,7 +76,8 @@ internal sealed class StaDispatcher : IStaDispatcher
|
||||
TimeSpan timeout,
|
||||
CancellationToken cancellationToken,
|
||||
Action<T>? abandonedResultCleanup = null,
|
||||
Action? abandonedFailureCleanup = null)
|
||||
Action? abandonedFailureCleanup = null,
|
||||
Action? preventFutureDispatch = null)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(callback);
|
||||
if (timeout <= TimeSpan.Zero)
|
||||
@@ -110,7 +112,7 @@ internal sealed class StaDispatcher : IStaDispatcher
|
||||
return await completion.ConfigureAwait(false);
|
||||
}
|
||||
|
||||
if (item.TryTimeout())
|
||||
if (item.TryTimeout(preventFutureDispatch))
|
||||
{
|
||||
Quarantine();
|
||||
}
|
||||
@@ -295,7 +297,7 @@ internal sealed class StaDispatcher : IStaDispatcher
|
||||
}
|
||||
}
|
||||
|
||||
public bool TryTimeout()
|
||||
public bool TryTimeout(Action? preventFutureDispatch)
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
@@ -307,6 +309,15 @@ internal sealed class StaDispatcher : IStaDispatcher
|
||||
|
||||
if (Interlocked.CompareExchange(ref _executionState, 2, state) == state)
|
||||
{
|
||||
try
|
||||
{
|
||||
preventFutureDispatch?.Invoke();
|
||||
}
|
||||
catch
|
||||
{
|
||||
// A timeout must still complete and quarantine the dispatcher.
|
||||
}
|
||||
|
||||
TimeoutCore();
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -19,6 +19,9 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
private readonly IStaDispatcher? _dispatcher;
|
||||
private readonly ILiveAuthorization _liveAuthorization;
|
||||
private readonly TimeProvider _timeProvider;
|
||||
private readonly Func<bool>? _finalConnectSafetyCheck;
|
||||
private readonly bool _abandonOnTargetMismatch;
|
||||
private readonly Action? _beforeSdkDispatchClaim;
|
||||
private readonly SemaphoreSlim _gate = new(1, 1);
|
||||
private readonly CancellationTokenSource _monitorCancellation = new();
|
||||
private readonly object _statusSync = new();
|
||||
@@ -36,6 +39,7 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
private int _lastKtapConnectState;
|
||||
private bool _connectionRequested;
|
||||
private volatile bool _outcomeUnknown;
|
||||
private volatile bool _quarantineIncomplete;
|
||||
private volatile bool _disposed;
|
||||
private int _disposeStarted;
|
||||
|
||||
@@ -47,7 +51,10 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
IStaDispatcher? dispatcher,
|
||||
ILiveAuthorization liveAuthorization,
|
||||
TimeProvider timeProvider,
|
||||
bool startMonitor = true)
|
||||
bool startMonitor = true,
|
||||
Func<bool>? finalConnectSafetyCheck = null,
|
||||
bool abandonOnTargetMismatch = false,
|
||||
Action? beforeSdkDispatchClaim = null)
|
||||
{
|
||||
_options = options;
|
||||
_sessionFactory = sessionFactory;
|
||||
@@ -56,6 +63,9 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
_dispatcher = dispatcher;
|
||||
_liveAuthorization = liveAuthorization;
|
||||
_timeProvider = timeProvider;
|
||||
_finalConnectSafetyCheck = finalConnectSafetyCheck;
|
||||
_abandonOnTargetMismatch = abandonOnTargetMismatch;
|
||||
_beforeSdkDispatchClaim = beforeSdkDispatchClaim;
|
||||
_connectionState = options.Mode switch
|
||||
{
|
||||
PlayoutMode.Disabled => PlayoutConnectionState.Disabled,
|
||||
@@ -135,9 +145,18 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
var session = _session;
|
||||
_session = null;
|
||||
var quarantineCompleted = !_quarantineIncomplete;
|
||||
if (session is not null)
|
||||
{
|
||||
await AbandonSessionOnStaAsync(session).ConfigureAwait(false);
|
||||
quarantineCompleted =
|
||||
await AbandonSessionOnStaAsync(session).ConfigureAwait(false) &&
|
||||
quarantineCompleted;
|
||||
}
|
||||
|
||||
if (!quarantineCompleted || _quarantineIncomplete)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
"The playout session could not be quarantined.");
|
||||
}
|
||||
|
||||
PublishStatus("송출 결과를 확인할 수 없어 연결을 격리했습니다. 출력 상태를 직접 확인하세요.");
|
||||
@@ -496,6 +515,7 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
PublishStatus(reconnecting ? "Tornado 테스트 송출에 다시 연결하는 중입니다." : "Tornado 송출에 연결하는 중입니다.");
|
||||
|
||||
IK3dSession? pendingSession = null;
|
||||
var dispatchLatch = new SdkDispatchLatch(_beforeSdkDispatchClaim);
|
||||
try
|
||||
{
|
||||
var session = _sessionFactory.Create();
|
||||
@@ -505,19 +525,53 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
{
|
||||
try
|
||||
{
|
||||
session.Connect(_options);
|
||||
if (_finalConnectSafetyCheck is null)
|
||||
{
|
||||
session.Connect(_options);
|
||||
}
|
||||
else if (session is IK3dGuardedConnectSession guardedSession)
|
||||
{
|
||||
guardedSession.ConnectGuarded(
|
||||
_options,
|
||||
IsExactTargetSafeForSdkDispatch,
|
||||
dispatchLatch);
|
||||
}
|
||||
else
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
return session;
|
||||
}
|
||||
catch
|
||||
{
|
||||
session.Dispose();
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
AbandonSessionInline(session);
|
||||
}
|
||||
else
|
||||
{
|
||||
session.Dispose();
|
||||
}
|
||||
|
||||
throw;
|
||||
}
|
||||
},
|
||||
_options.ConnectTimeout,
|
||||
cancellationToken,
|
||||
static lateSession => lateSession.Dispose(),
|
||||
() => Volatile.Read(ref pendingSession)?.TryPreventKtapConnect()).ConfigureAwait(false);
|
||||
lateSession =>
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
AbandonSessionInline(lateSession);
|
||||
}
|
||||
else
|
||||
{
|
||||
lateSession.Dispose();
|
||||
}
|
||||
},
|
||||
() => Volatile.Read(ref pendingSession)?.TryPreventKtapConnect(),
|
||||
dispatchLatch.PreventFutureDispatch).ConfigureAwait(false);
|
||||
|
||||
CaptureKtapEvidence(_session);
|
||||
|
||||
@@ -525,6 +579,12 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
if (!IsProcessEligible(_processSnapshot) ||
|
||||
!processSnapshotBeforeConnect.HasSameProcessGeneration(_processSnapshot))
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
await AbandonCurrentSessionAsync().ConfigureAwait(false);
|
||||
return MarkOutcomeUnknown(PlayoutOperation.Connect);
|
||||
}
|
||||
|
||||
if (!await ReleaseSessionAsync(CancellationToken.None).ConfigureAwait(false))
|
||||
{
|
||||
return MarkOutcomeUnknown(PlayoutOperation.Disconnect);
|
||||
@@ -557,25 +617,52 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
PreventLateKtapDispatchAndCapture(Volatile.Read(ref pendingSession));
|
||||
var cancelledSession = Volatile.Read(ref pendingSession);
|
||||
PreventLateKtapDispatchAndCapture(cancelledSession);
|
||||
if (_abandonOnTargetMismatch &&
|
||||
cancelledSession is not null &&
|
||||
cancelledSession.LastKtapConnectState != PlayoutKtapConnectState.NotAttempted)
|
||||
{
|
||||
_session = null;
|
||||
return MarkOutcomeUnknown(PlayoutOperation.Connect);
|
||||
}
|
||||
|
||||
_connectionState = PlayoutConnectionState.Disconnected;
|
||||
PublishStatus("송출 연결 요청이 취소되었습니다.");
|
||||
return Result(PlayoutOperation.Connect, PlayoutResultCode.Cancelled, "송출 요청이 취소되었습니다.");
|
||||
}
|
||||
catch (StaOperationTimedOutException)
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
PreventLateKtapDispatchAndCapture(Volatile.Read(ref pendingSession));
|
||||
return MarkOutcomeUnknown(PlayoutOperation.Connect);
|
||||
}
|
||||
catch (StaDispatcherQuarantinedException)
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
PreventLateKtapDispatchAndCapture(Volatile.Read(ref pendingSession));
|
||||
return MarkOutcomeUnknown(PlayoutOperation.Connect);
|
||||
}
|
||||
catch (Exception)
|
||||
{
|
||||
CaptureKtapEvidence(Volatile.Read(ref pendingSession));
|
||||
var failedSession = Volatile.Read(ref pendingSession);
|
||||
CaptureKtapEvidence(failedSession);
|
||||
_session = null;
|
||||
if (_abandonOnTargetMismatch &&
|
||||
failedSession is not null &&
|
||||
failedSession.LastKtapConnectState != PlayoutKtapConnectState.NotAttempted)
|
||||
{
|
||||
return MarkOutcomeUnknown(PlayoutOperation.Connect);
|
||||
}
|
||||
|
||||
ScheduleReconnect();
|
||||
_connectionState = reconnecting
|
||||
? PlayoutConnectionState.Reconnecting
|
||||
@@ -683,7 +770,9 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
var result = await RunSessionOperationAsync(
|
||||
PlayoutOperation.Prepare,
|
||||
session => session.Prepare(cue, _options.LayoutIndex),
|
||||
(session, dispatchLatch) => ExecuteSdkDispatch(
|
||||
dispatchLatch,
|
||||
() => session.Prepare(cue, _options.LayoutIndex)),
|
||||
partialOutcomeUnknown: false,
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
if (result.IsSuccess)
|
||||
@@ -736,7 +825,9 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
var prepared = _preparedCue;
|
||||
var result = await RunSessionOperationAsync(
|
||||
PlayoutOperation.TakeIn,
|
||||
session => session.Play(_options.LayoutIndex),
|
||||
(session, dispatchLatch) => ExecuteSdkDispatch(
|
||||
dispatchLatch,
|
||||
() => session.Play(_options.LayoutIndex)),
|
||||
partialOutcomeUnknown: true,
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
if (result.IsSuccess)
|
||||
@@ -802,10 +893,14 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
var result = await RunSessionOperationAsync(
|
||||
PlayoutOperation.Next,
|
||||
session =>
|
||||
(session, dispatchLatch) =>
|
||||
{
|
||||
session.Prepare(resolved, _options.LayoutIndex);
|
||||
session.Play(_options.LayoutIndex);
|
||||
ExecuteSdkDispatch(
|
||||
dispatchLatch,
|
||||
() => session.Prepare(resolved, _options.LayoutIndex));
|
||||
ExecuteSdkDispatch(
|
||||
dispatchLatch,
|
||||
() => session.Play(_options.LayoutIndex));
|
||||
},
|
||||
partialOutcomeUnknown: true,
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
@@ -856,7 +951,9 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
var result = await RunSessionOperationAsync(
|
||||
PlayoutOperation.TakeOut,
|
||||
session => session.TakeOut(_options.LayoutIndex, scope),
|
||||
(session, dispatchLatch) => ExecuteSdkDispatch(
|
||||
dispatchLatch,
|
||||
() => session.TakeOut(_options.LayoutIndex, scope)),
|
||||
partialOutcomeUnknown: true,
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
if (result.IsSuccess)
|
||||
@@ -881,6 +978,12 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
_processSnapshot = CaptureProcessSnapshot();
|
||||
if (!IsProcessEligible(_processSnapshot))
|
||||
{
|
||||
if (_abandonOnTargetMismatch && _session is not null)
|
||||
{
|
||||
await AbandonCurrentSessionAsync().ConfigureAwait(false);
|
||||
return MarkOutcomeUnknown(operation);
|
||||
}
|
||||
|
||||
if (_onAirSceneName is not null)
|
||||
{
|
||||
await AbandonCurrentSessionAsync().ConfigureAwait(false);
|
||||
@@ -904,6 +1007,12 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
if (HasConnectedProcessGenerationChanged(_processSnapshot))
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
await AbandonCurrentSessionAsync().ConfigureAwait(false);
|
||||
return MarkOutcomeUnknown(operation);
|
||||
}
|
||||
|
||||
if (_onAirSceneName is not null)
|
||||
{
|
||||
await AbandonCurrentSessionAsync().ConfigureAwait(false);
|
||||
@@ -938,18 +1047,22 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
private async Task<PlayoutResult> RunSessionOperationAsync(
|
||||
PlayoutOperation operation,
|
||||
Action<IK3dSession> callback,
|
||||
Action<IK3dSession, SdkDispatchLatch> callback,
|
||||
bool partialOutcomeUnknown,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var session = _session!;
|
||||
var dispatchLatch = new SdkDispatchLatch(_beforeSdkDispatchClaim);
|
||||
var callbackStarted = 0;
|
||||
var preserveActiveOutput = partialOutcomeUnknown || _onAirSceneName is not null;
|
||||
var preserveActiveOutput =
|
||||
_abandonOnTargetMismatch ||
|
||||
partialOutcomeUnknown ||
|
||||
_onAirSceneName is not null;
|
||||
void CleanupAbandonedOperation()
|
||||
{
|
||||
if (preserveActiveOutput)
|
||||
{
|
||||
session.Abandon();
|
||||
AbandonSessionInline(session);
|
||||
}
|
||||
else
|
||||
{
|
||||
@@ -965,7 +1078,7 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
Volatile.Write(ref callbackStarted, 1);
|
||||
try
|
||||
{
|
||||
callback(session);
|
||||
callback(session, dispatchLatch);
|
||||
}
|
||||
finally
|
||||
{
|
||||
@@ -980,7 +1093,8 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
_options.OperationTimeout,
|
||||
cancellationToken,
|
||||
_ => CleanupAbandonedOperation(),
|
||||
CleanupAbandonedOperation).ConfigureAwait(false);
|
||||
CleanupAbandonedOperation,
|
||||
dispatchLatch.PreventFutureDispatch).ConfigureAwait(false);
|
||||
return Result(operation, PlayoutResultCode.Success, SuccessMessage(operation));
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
@@ -997,10 +1111,20 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
}
|
||||
catch (StaOperationTimedOutException)
|
||||
{
|
||||
if (preserveActiveOutput)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
return MarkOutcomeUnknown(operation);
|
||||
}
|
||||
catch (StaDispatcherQuarantinedException)
|
||||
{
|
||||
if (preserveActiveOutput)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
return MarkOutcomeUnknown(operation);
|
||||
}
|
||||
catch (Exception)
|
||||
@@ -1062,6 +1186,11 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
if (_dispatcher is null || _dispatcher.IsQuarantined)
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -1074,6 +1203,11 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
{
|
||||
if (_dispatcher.IsQuarantined)
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -1084,11 +1218,21 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
}
|
||||
catch (Exception)
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
}
|
||||
catch (Exception)
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1097,6 +1241,7 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
{
|
||||
if (_dispatcher is null || _dispatcher.IsQuarantined)
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -1105,18 +1250,19 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
await _dispatcher.InvokeAsync(
|
||||
() =>
|
||||
{
|
||||
session.Abandon();
|
||||
AbandonSessionInline(session);
|
||||
return true;
|
||||
},
|
||||
_options.DisconnectTimeout,
|
||||
CancellationToken.None,
|
||||
_ => session.Abandon(),
|
||||
() => session.Abandon()).ConfigureAwait(false);
|
||||
_ => AbandonSessionInline(session),
|
||||
() => AbandonSessionInline(session)).ConfigureAwait(false);
|
||||
return true;
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Never fall back to Disconnect when the physical output state is unknown.
|
||||
_quarantineIncomplete = true;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1134,13 +1280,39 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
|
||||
private Task<bool> CleanupSessionOnStaAsync(
|
||||
IK3dSession session,
|
||||
CancellationToken cancellationToken) =>
|
||||
_dispatcher!.InvokeAsync(
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var dispatchLatch = new SdkDispatchLatch(_beforeSdkDispatchClaim);
|
||||
return _dispatcher!.InvokeAsync(
|
||||
() =>
|
||||
{
|
||||
if (_abandonOnTargetMismatch)
|
||||
{
|
||||
var disconnected = false;
|
||||
try
|
||||
{
|
||||
ExecuteSdkDispatch(dispatchLatch, session.Disconnect);
|
||||
disconnected = true;
|
||||
}
|
||||
catch
|
||||
{
|
||||
AbandonSessionInline(session);
|
||||
throw;
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (disconnected)
|
||||
{
|
||||
session.Dispose();
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
session.Disconnect();
|
||||
ExecuteSdkDispatch(dispatchLatch, session.Disconnect);
|
||||
}
|
||||
finally
|
||||
{
|
||||
@@ -1150,7 +1322,130 @@ internal sealed class TornadoPlayoutEngine : IPlayoutEngine
|
||||
return true;
|
||||
},
|
||||
_options.DisconnectTimeout,
|
||||
cancellationToken);
|
||||
cancellationToken,
|
||||
preventFutureDispatch: dispatchLatch.PreventFutureDispatch);
|
||||
}
|
||||
|
||||
private void EnsureExactTarget()
|
||||
{
|
||||
if (_finalConnectSafetyCheck is not null &&
|
||||
!IsExactTargetSafeForSdkDispatch())
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
}
|
||||
|
||||
private void ExecuteSdkDispatch(
|
||||
SdkDispatchLatch dispatchLatch,
|
||||
Action sdkCall)
|
||||
{
|
||||
EnsureExactTarget();
|
||||
using var dispatchClaim = dispatchLatch.TryClaim();
|
||||
if (dispatchClaim is null)
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
sdkCall();
|
||||
EnsureExactTarget();
|
||||
}
|
||||
|
||||
private bool IsExactTargetSafeForSdkDispatch()
|
||||
{
|
||||
var dispatcher = _dispatcher;
|
||||
if (dispatcher is null || dispatcher.IsQuarantined)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
return _finalConnectSafetyCheck is not null &&
|
||||
_finalConnectSafetyCheck() &&
|
||||
!dispatcher.IsQuarantined;
|
||||
}
|
||||
catch
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private void AbandonSessionInline(IK3dSession session)
|
||||
{
|
||||
try
|
||||
{
|
||||
session.Abandon();
|
||||
}
|
||||
catch
|
||||
{
|
||||
_quarantineIncomplete = true;
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class SdkDispatchLatch(Action? beforeClaim) : IK3dSdkDispatchLatch
|
||||
{
|
||||
private const int Open = 0;
|
||||
private const int Claimed = 1;
|
||||
private const int Prevented = 2;
|
||||
private const int ClaimedAndPrevented = 3;
|
||||
|
||||
private int _state;
|
||||
|
||||
public IDisposable? TryClaim()
|
||||
{
|
||||
beforeClaim?.Invoke();
|
||||
return Interlocked.CompareExchange(ref _state, Claimed, Open) == Open
|
||||
? new DispatchClaim(this)
|
||||
: null;
|
||||
}
|
||||
|
||||
public void PreventFutureDispatch()
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var state = Volatile.Read(ref _state);
|
||||
var next = state switch
|
||||
{
|
||||
Open => Prevented,
|
||||
Claimed => ClaimedAndPrevented,
|
||||
_ => state
|
||||
};
|
||||
if (next == state ||
|
||||
Interlocked.CompareExchange(ref _state, next, state) == state)
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void Release()
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var state = Volatile.Read(ref _state);
|
||||
var next = state switch
|
||||
{
|
||||
Claimed => Open,
|
||||
ClaimedAndPrevented => Prevented,
|
||||
_ => state
|
||||
};
|
||||
if (next == state ||
|
||||
Interlocked.CompareExchange(ref _state, next, state) == state)
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class DispatchClaim(SdkDispatchLatch owner) : IDisposable
|
||||
{
|
||||
private SdkDispatchLatch? _owner = owner;
|
||||
|
||||
public void Dispose() =>
|
||||
Interlocked.Exchange(ref _owner, null)?.Release();
|
||||
}
|
||||
}
|
||||
|
||||
private async Task MonitorLoopAsync()
|
||||
{
|
||||
|
||||
@@ -185,6 +185,95 @@ public sealed class DynamicK3dSessionTests
|
||||
Assert.Equal(1, log.Names.Count(name => name == "Disconnect"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_WhenFinalTargetCheckFails_BlocksKtapAndNeverDisconnects()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var options = ValidatedPlayoutOptions.Create(TestOptions(scenes.Path));
|
||||
var log = new FakeComLog();
|
||||
var engine = new FakeEngine(log, new FakePlayer(log), new FakeScene(log));
|
||||
var activator = new FakeActivator(log, engine);
|
||||
var releaser = new FakeReleaser(
|
||||
log,
|
||||
(engine, "Engine"),
|
||||
(activator.EventHandler, "EventHandler"));
|
||||
await using var dispatcher = new StaDispatcher(capacity: 1);
|
||||
|
||||
await dispatcher.InvokeAsync(
|
||||
() =>
|
||||
{
|
||||
using var session = new DynamicK3dSession(activator, releaser);
|
||||
Assert.Throws<K3dConnectSafetyException>(() => session.ConnectGuarded(
|
||||
options,
|
||||
() =>
|
||||
{
|
||||
log.Add("FinalSafetyCheck");
|
||||
return false;
|
||||
},
|
||||
new FakeK3dSdkDispatchLatch()));
|
||||
Assert.Equal(
|
||||
PlayoutKtapConnectState.NotAttempted,
|
||||
session.LastKtapConnectState);
|
||||
return true;
|
||||
},
|
||||
TimeSpan.FromSeconds(5),
|
||||
CancellationToken.None);
|
||||
|
||||
Assert.Equal(
|
||||
new[]
|
||||
{
|
||||
$"Create:{K3dComConstants.KaEventHandlerClassGuid:B}",
|
||||
$"Create:{K3dComConstants.KaEngineClassGuid:B}",
|
||||
"FinalSafetyCheck",
|
||||
"Release:Engine",
|
||||
"Release:EventHandler"
|
||||
},
|
||||
log.Names);
|
||||
Assert.DoesNotContain(log.Names, name =>
|
||||
name.StartsWith("KTAPConnect:", StringComparison.Ordinal));
|
||||
Assert.DoesNotContain("Disconnect", log.Names);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_WhenKtapThrows_ReleasesWithoutSdkDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var options = ValidatedPlayoutOptions.Create(TestOptions(scenes.Path));
|
||||
var log = new FakeComLog();
|
||||
var failure = new COMException("fake ambiguous guarded connect failure");
|
||||
var engine = new FakeEngine(
|
||||
log,
|
||||
new FakePlayer(log),
|
||||
new FakeScene(log),
|
||||
connectFailure: failure);
|
||||
var activator = new FakeActivator(log, engine);
|
||||
var releaser = new FakeReleaser(
|
||||
log,
|
||||
(engine, "Engine"),
|
||||
(activator.EventHandler, "EventHandler"));
|
||||
await using var dispatcher = new StaDispatcher(capacity: 1);
|
||||
|
||||
await dispatcher.InvokeAsync(
|
||||
() =>
|
||||
{
|
||||
using var session = new DynamicK3dSession(activator, releaser);
|
||||
var exception = Assert.Throws<COMException>(
|
||||
() => session.ConnectGuarded(
|
||||
options,
|
||||
() => true,
|
||||
new FakeK3dSdkDispatchLatch()));
|
||||
Assert.Same(failure, exception);
|
||||
return true;
|
||||
},
|
||||
TimeSpan.FromSeconds(5),
|
||||
CancellationToken.None);
|
||||
|
||||
Assert.Contains("KTAPConnect:1:127.0.0.1:30001:0", log.Names);
|
||||
Assert.DoesNotContain("Disconnect", log.Names);
|
||||
Assert.Equal(1, log.Names.Count(name => name == "Release:Engine"));
|
||||
Assert.Equal(1, log.Names.Count(name => name == "Release:EventHandler"));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(PlayoutTakeOutScope.All, "StopAll")]
|
||||
[InlineData(PlayoutTakeOutScope.Layout, "CutOut:10")]
|
||||
|
||||
@@ -0,0 +1,863 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.Json;
|
||||
using MBN_STOCK_WEBVIEW.Core.Playout;
|
||||
using MBN_STOCK_WEBVIEW.Playout;
|
||||
using MBN_STOCK_WEBVIEW.Playout.Configuration;
|
||||
using MBN_STOCK_WEBVIEW.Playout.Diagnostics;
|
||||
using MBN_STOCK_WEBVIEW.Playout.Runtime;
|
||||
|
||||
namespace MBN_STOCK_WEBVIEW.Playout.Tests;
|
||||
|
||||
public sealed class PgmCutsSequenceRunnerTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task StableTarget_ExecutesFixedSequenceWithExactSafetyAndMilestoneOrder()
|
||||
{
|
||||
using var environment = new EnvironmentScope()
|
||||
.Set("MBN_STOCK_PLAYOUT_MODE", "DryRun")
|
||||
.Set("MBN_STOCK_PLAYOUT_HOST", "192.0.2.99")
|
||||
.Set("MBN_STOCK_PLAYOUT_OUTPUT_CHANNEL", "77")
|
||||
.Set("MBN_STOCK_PLAYOUT_AUTHORIZE_LIVE_OUTPUT", "false");
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var safety = new FakeSafetyProbe(events);
|
||||
var engine = new FakeEngine(events);
|
||||
var factory = new FakeEngineFactory(events, engine);
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
safety,
|
||||
new FakeHashPins(true),
|
||||
factory,
|
||||
new FakeDelay(events),
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
var report = await runner.ExecuteAsync(
|
||||
Request(scenes.Path),
|
||||
stage => events.Enqueue($"Milestone:{stage}"));
|
||||
|
||||
Assert.True(report.Completed);
|
||||
Assert.False(report.OutcomeUnknown);
|
||||
Assert.False(report.QuarantineAttempted);
|
||||
Assert.Equal(
|
||||
new[]
|
||||
{
|
||||
"Safety:1",
|
||||
"CreateEngine",
|
||||
"Safety:2",
|
||||
"Connect",
|
||||
"Safety:3",
|
||||
"Safety:4",
|
||||
"Prepare:5001",
|
||||
"Safety:5",
|
||||
"Safety:6",
|
||||
"TakeIn",
|
||||
"Safety:7",
|
||||
"Safety:8",
|
||||
"Milestone:First",
|
||||
"Delay:5000",
|
||||
"Safety:9",
|
||||
"Safety:10",
|
||||
"Next:5006",
|
||||
"Safety:11",
|
||||
"Safety:12",
|
||||
"Milestone:Next",
|
||||
"Delay:5000",
|
||||
"Safety:13",
|
||||
"Safety:14",
|
||||
"TakeOut:All",
|
||||
"Safety:15",
|
||||
"Safety:16",
|
||||
"Milestone:TakeOut",
|
||||
"Delay:5000",
|
||||
"Safety:17",
|
||||
"Safety:18",
|
||||
"Disconnect",
|
||||
"Safety:19",
|
||||
"Dispose"
|
||||
},
|
||||
events);
|
||||
Assert.Equal(
|
||||
new[]
|
||||
{
|
||||
PgmCutsSequenceStage.Connect,
|
||||
PgmCutsSequenceStage.PrepareFirst,
|
||||
PgmCutsSequenceStage.TakeIn,
|
||||
PgmCutsSequenceStage.Next,
|
||||
PgmCutsSequenceStage.TakeOut,
|
||||
PgmCutsSequenceStage.Disconnect
|
||||
},
|
||||
report.Commands.Select(command => command.Stage));
|
||||
|
||||
var plan = Assert.IsType<PgmCutsSequencePlan>(factory.Plan);
|
||||
Assert.Equal(PlayoutMode.Live, plan.Options.Mode);
|
||||
Assert.Equal("127.0.0.1", plan.Options.Host);
|
||||
Assert.Equal(30_001, plan.Options.Port);
|
||||
Assert.Null(plan.Options.OutputChannel);
|
||||
Assert.Equal(10, plan.Options.LayoutIndex);
|
||||
Assert.True(plan.Options.TrustedLiveOutputEnabled);
|
||||
Assert.False(plan.Options.ReconnectEnabled);
|
||||
Assert.Equal(0, plan.Options.MaximumReconnectAttempts);
|
||||
Assert.Equal(TimeSpan.FromMilliseconds(5_000), plan.ObservationDuration);
|
||||
Assert.Equal("5001", plan.FirstCue.SceneName);
|
||||
Assert.Equal("5006", plan.NextCue.SceneName);
|
||||
Assert.Equal("5001.t2s", plan.FirstCue.SceneFile);
|
||||
Assert.Equal("5006.t2s", plan.NextCue.SceneFile);
|
||||
Assert.False(Path.IsPathFullyQualified(plan.FirstCue.SceneFile));
|
||||
Assert.False(Path.IsPathFullyQualified(plan.NextCue.SceneFile));
|
||||
Assert.True(Path.IsPathFullyQualified(plan.FirstAssetPath));
|
||||
Assert.True(Path.IsPathFullyQualified(plan.NextAssetPath));
|
||||
Assert.Equal("5001.t2s", Path.GetFileName(plan.FirstAssetPath));
|
||||
Assert.Equal("5006.t2s", Path.GetFileName(plan.NextAssetPath));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RawPlanCue_IsResolvedExactlyOnceByTornadoEngineBeforeSdkPrepare()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var captureFactory = new FakeEngineFactory(events, new FakeEngine(events));
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(events),
|
||||
new FakeHashPins(true),
|
||||
captureFactory,
|
||||
new FakeDelay(events),
|
||||
new FakeAssetLeases(true));
|
||||
Assert.True((await runner.ExecuteAsync(Request(scenes.Path))).Completed);
|
||||
var plan = Assert.IsType<PgmCutsSequencePlan>(captureFactory.Plan);
|
||||
PlayoutCue? sdkCue = null;
|
||||
var session = new FakeK3dSession
|
||||
{
|
||||
PrepareAction = (cue, _) => sdkCue = cue
|
||||
};
|
||||
var dispatcher = new StaDispatcher(plan.Options.QueueCapacity);
|
||||
await using var engine = new TornadoPlayoutEngine(
|
||||
plan.Options,
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
dispatcher,
|
||||
new FakeLiveAuthorization(true),
|
||||
TimeProvider.System,
|
||||
startMonitor: false);
|
||||
|
||||
Assert.False(Path.IsPathFullyQualified(plan.FirstCue.SceneFile));
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
|
||||
var prepare = await engine.PrepareAsync(
|
||||
plan.FirstCue,
|
||||
CancellationToken.None);
|
||||
|
||||
Assert.True(prepare.IsSuccess);
|
||||
var resolvedCue = Assert.IsType<PlayoutCue>(sdkCue);
|
||||
Assert.True(Path.IsPathFullyQualified(resolvedCue.SceneFile));
|
||||
Assert.Equal(
|
||||
Path.GetFullPath(plan.FirstAssetPath),
|
||||
Path.GetFullPath(resolvedCue.SceneFile),
|
||||
StringComparer.OrdinalIgnoreCase);
|
||||
Assert.True((await engine.DisconnectAsync(CancellationToken.None)).IsSuccess);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(2)]
|
||||
[InlineData(3)]
|
||||
[InlineData(4)]
|
||||
[InlineData(5)]
|
||||
[InlineData(6)]
|
||||
[InlineData(7)]
|
||||
[InlineData(8)]
|
||||
[InlineData(9)]
|
||||
[InlineData(10)]
|
||||
[InlineData(11)]
|
||||
[InlineData(12)]
|
||||
[InlineData(13)]
|
||||
[InlineData(14)]
|
||||
[InlineData(15)]
|
||||
[InlineData(16)]
|
||||
[InlineData(17)]
|
||||
[InlineData(18)]
|
||||
[InlineData(19)]
|
||||
public async Task TargetChangeAtEveryBoundary_StopsAndQuarantinesWithoutLaterOutput(
|
||||
int changedCapture)
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var engine = new FakeEngine(events);
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(events, changedCapture),
|
||||
new FakeHashPins(true),
|
||||
new FakeEngineFactory(events, engine),
|
||||
new FakeDelay(events),
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
var report = await runner.ExecuteAsync(
|
||||
Request(scenes.Path),
|
||||
stage => events.Enqueue($"Milestone:{stage}"));
|
||||
|
||||
Assert.False(report.Completed);
|
||||
Assert.Equal("pgm-target-changed", report.ErrorCode);
|
||||
Assert.True(report.QuarantineAttempted);
|
||||
Assert.True(report.QuarantineCompleted);
|
||||
Assert.Equal(1, events.Count(value => value == "Quarantine"));
|
||||
var quarantineIndex = Array.IndexOf(events.ToArray(), "Quarantine");
|
||||
Assert.DoesNotContain(
|
||||
events.Skip(quarantineIndex + 1),
|
||||
value => IsOutputCommand(value) || value == "Disconnect");
|
||||
if (changedCapture <= 18)
|
||||
{
|
||||
Assert.DoesNotContain("Disconnect", events);
|
||||
}
|
||||
|
||||
Assert.Equal(changedCapture is > 2 and < 19, report.OutcomeUnknown);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData((int)PgmCutsSequenceStage.Connect)]
|
||||
[InlineData((int)PgmCutsSequenceStage.PrepareFirst)]
|
||||
[InlineData((int)PgmCutsSequenceStage.TakeIn)]
|
||||
[InlineData((int)PgmCutsSequenceStage.Next)]
|
||||
[InlineData((int)PgmCutsSequenceStage.TakeOut)]
|
||||
[InlineData((int)PgmCutsSequenceStage.Disconnect)]
|
||||
public async Task OutcomeUnknownAtEveryCommand_QuarantinesWithoutSubsequentCommand(
|
||||
int stageValue)
|
||||
{
|
||||
var stage = (PgmCutsSequenceStage)stageValue;
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var engine = new FakeEngine(events);
|
||||
engine.Results[stage] = PlayoutResultCode.OutcomeUnknown;
|
||||
var runner = Runner(events, engine);
|
||||
|
||||
var report = await runner.ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.False(report.Completed);
|
||||
Assert.True(report.OutcomeUnknown);
|
||||
Assert.True(report.QuarantineCompleted);
|
||||
Assert.Equal("command-outcome-unknown", report.ErrorCode);
|
||||
var quarantineIndex = Array.IndexOf(events.ToArray(), "Quarantine");
|
||||
Assert.DoesNotContain(
|
||||
events.Skip(quarantineIndex + 1),
|
||||
value => IsOutputCommand(value) || value == "Disconnect");
|
||||
if (stage != PgmCutsSequenceStage.Disconnect)
|
||||
{
|
||||
Assert.DoesNotContain("Disconnect", events);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task KnownPrepareFailure_PerformsExplicitSameTargetDisconnectCleanup()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var engine = new FakeEngine(events);
|
||||
engine.Results[PgmCutsSequenceStage.PrepareFirst] = PlayoutResultCode.Failed;
|
||||
|
||||
var report = await Runner(events, engine).ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.False(report.Completed);
|
||||
Assert.False(report.OutcomeUnknown);
|
||||
Assert.False(report.QuarantineAttempted);
|
||||
Assert.True(report.DisconnectAttempted);
|
||||
Assert.Equal(
|
||||
new[] { "Connect", "Prepare:5001", "Disconnect", "Dispose" },
|
||||
EngineCalls(events));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task QuarantineFailure_SkipsDisposeAndEveryFurtherSdkCommand()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var engine = new FakeEngine(events)
|
||||
{
|
||||
QuarantineFailure = new InvalidOperationException("fake quarantine failure")
|
||||
};
|
||||
engine.Results[PgmCutsSequenceStage.Next] = PlayoutResultCode.OutcomeUnknown;
|
||||
|
||||
var report = await Runner(events, engine).ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.True(report.QuarantineAttempted);
|
||||
Assert.False(report.QuarantineCompleted);
|
||||
Assert.True(report.OutcomeUnknown);
|
||||
Assert.DoesNotContain("Dispose", events);
|
||||
Assert.DoesNotContain("Disconnect", events);
|
||||
Assert.Equal("Quarantine", events.Last());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TimedOutTakeOut_IsTreatedAsUnknownAndNeverDisconnects()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var engine = new FakeEngine(events);
|
||||
engine.Results[PgmCutsSequenceStage.TakeOut] = PlayoutResultCode.TimedOut;
|
||||
|
||||
var report = await Runner(events, engine).ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.True(report.RenderCommandAttempted);
|
||||
Assert.True(report.OutcomeUnknown);
|
||||
Assert.True(report.QuarantineCompleted);
|
||||
Assert.DoesNotContain("Disconnect", events);
|
||||
Assert.Equal(1, events.Count(value => value == "TakeOut:All"));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(1, false)]
|
||||
[InlineData(2, true)]
|
||||
public async Task InterruptedOnAirObservation_PerformsOneShotTakeOutThenDisconnect(
|
||||
int interruptedObservation,
|
||||
bool nextWasRendered)
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var delay = new FakeDelay(events)
|
||||
{
|
||||
FailureOnCall = interruptedObservation
|
||||
};
|
||||
var engine = new FakeEngine(events);
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(events),
|
||||
new FakeHashPins(true),
|
||||
new FakeEngineFactory(events, engine),
|
||||
delay,
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
var report = await runner.ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.False(report.Completed);
|
||||
Assert.False(report.OutcomeUnknown);
|
||||
Assert.False(report.QuarantineAttempted);
|
||||
Assert.Equal("observation-interrupted", report.ErrorCode);
|
||||
Assert.Equal(1, events.Count(value => value == "TakeOut:All"));
|
||||
Assert.Equal(1, events.Count(value => value == "Disconnect"));
|
||||
Assert.Equal(nextWasRendered, events.Contains("Next:5006"));
|
||||
Assert.Equal(
|
||||
new[] { PgmCutsSequenceStage.TakeOut, PgmCutsSequenceStage.Disconnect },
|
||||
report.Commands.TakeLast(2).Select(command => command.Stage));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(PlayoutResultCode.Failed)]
|
||||
[InlineData(PlayoutResultCode.TimedOut)]
|
||||
[InlineData(PlayoutResultCode.OutcomeUnknown)]
|
||||
public async Task InterruptedObservation_WhenOneShotTakeOutIsNotSuccessful_QuarantinesWithoutDisconnect(
|
||||
PlayoutResultCode takeOutCode)
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var engine = new FakeEngine(events);
|
||||
engine.Results[PgmCutsSequenceStage.TakeOut] = takeOutCode;
|
||||
var delay = new FakeDelay(events)
|
||||
{
|
||||
FailureOnCall = 1
|
||||
};
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(events),
|
||||
new FakeHashPins(true),
|
||||
new FakeEngineFactory(events, engine),
|
||||
delay,
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
var report = await runner.ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.False(report.Completed);
|
||||
Assert.True(report.OutcomeUnknown);
|
||||
Assert.True(report.QuarantineCompleted);
|
||||
Assert.Equal(1, events.Count(value => value == "TakeOut:All"));
|
||||
Assert.DoesNotContain("Disconnect", events);
|
||||
var takeOutIndex = Array.IndexOf(events.ToArray(), "TakeOut:All");
|
||||
Assert.DoesNotContain(
|
||||
events.Skip(takeOutIndex + 1),
|
||||
value => value == "TakeOut:All");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task InterruptedPostTakeOutObservation_DoesNotRepeatTakeOutAndDisconnects()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var delay = new FakeDelay(events)
|
||||
{
|
||||
FailureOnCall = 3
|
||||
};
|
||||
var engine = new FakeEngine(events);
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(events),
|
||||
new FakeHashPins(true),
|
||||
new FakeEngineFactory(events, engine),
|
||||
delay,
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
var report = await runner.ExecuteAsync(Request(scenes.Path));
|
||||
|
||||
Assert.False(report.Completed);
|
||||
Assert.False(report.OutcomeUnknown);
|
||||
Assert.False(report.QuarantineAttempted);
|
||||
Assert.Equal("observation-interrupted", report.ErrorCode);
|
||||
Assert.Equal(1, events.Count(value => value == "TakeOut:All"));
|
||||
Assert.Equal(1, events.Count(value => value == "Disconnect"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task MissingPinsOrInvalidAssets_RejectBeforeEngineCreation()
|
||||
{
|
||||
using var missingScene = TemporarySceneDirectory.Create("5001.t2s");
|
||||
using var validScenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var missingPinEvents = new ConcurrentQueue<string>();
|
||||
var pinFactory = new FakeEngineFactory(missingPinEvents, new FakeEngine(missingPinEvents));
|
||||
var missingPins = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(missingPinEvents),
|
||||
new FakeHashPins(false),
|
||||
pinFactory,
|
||||
new FakeDelay(missingPinEvents));
|
||||
var assetEvents = new ConcurrentQueue<string>();
|
||||
var assetFactory = new FakeEngineFactory(assetEvents, new FakeEngine(assetEvents));
|
||||
var invalidAssets = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(assetEvents),
|
||||
new FakeHashPins(true),
|
||||
assetFactory,
|
||||
new FakeDelay(assetEvents),
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
var rejectedAssetEvents = new ConcurrentQueue<string>();
|
||||
var rejectedAssetFactory = new FakeEngineFactory(
|
||||
rejectedAssetEvents,
|
||||
new FakeEngine(rejectedAssetEvents));
|
||||
var rejectedAssets = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(rejectedAssetEvents),
|
||||
new FakeHashPins(true),
|
||||
rejectedAssetFactory,
|
||||
new FakeDelay(rejectedAssetEvents),
|
||||
new FakeAssetLeases(false));
|
||||
|
||||
var pinReport = await missingPins.ExecuteAsync(Request(validScenes.Path));
|
||||
var assetReport = await invalidAssets.ExecuteAsync(Request(missingScene.Path));
|
||||
var rejectedAssetReport = await rejectedAssets.ExecuteAsync(Request(validScenes.Path));
|
||||
|
||||
Assert.Equal("hash-pins-required", pinReport.ErrorCode);
|
||||
Assert.Equal("request-rejected", assetReport.ErrorCode);
|
||||
Assert.Equal("cut-assets-not-approved", rejectedAssetReport.ErrorCode);
|
||||
Assert.Equal(0, pinFactory.CreateCount);
|
||||
Assert.Equal(0, assetFactory.CreateCount);
|
||||
Assert.Equal(0, rejectedAssetFactory.CreateCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ReportAndMilestonesContainNoEndpointTitleIdentityPathHashOrSceneCode()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
const string title = "Unique PGM Private Title";
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var report = await Runner(events, new FakeEngine(events)).ExecuteAsync(
|
||||
Request(scenes.Path) with { ExpectedPgmWindowTitle = title });
|
||||
|
||||
var json = JsonSerializer.Serialize(report);
|
||||
Assert.DoesNotContain("127.0.0.1", json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain("30001", json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain(title, json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain(scenes.Path, json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain("4242", json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain(new string('A', 64), json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain("5001", json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.DoesNotContain("5006", json, StringComparison.OrdinalIgnoreCase);
|
||||
Assert.All(
|
||||
events.Where(value => value.StartsWith("Milestone:", StringComparison.Ordinal)),
|
||||
value => Assert.DoesNotContain("500", value, StringComparison.Ordinal));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RunnerCapabilityAndEntryPointsAreNotPublic_AndNormalTestGateIsUnchanged()
|
||||
{
|
||||
Assert.False(typeof(PgmCutsSequenceRunner).IsPublic);
|
||||
Assert.False(typeof(PgmCutsSequenceRequest).IsPublic);
|
||||
Assert.False(typeof(PgmCutsSequenceReport).IsPublic);
|
||||
Assert.All(
|
||||
typeof(FullPgmCutsSequenceEngineFactory).GetNestedTypes(
|
||||
System.Reflection.BindingFlags.Public |
|
||||
System.Reflection.BindingFlags.NonPublic),
|
||||
type => Assert.False(type.IsPublic));
|
||||
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var normalTest = new PlayoutOptions
|
||||
{
|
||||
Mode = PlayoutMode.Test,
|
||||
SceneDirectory = scenes.Path,
|
||||
OutputChannel = 9,
|
||||
TestProcessWindowTitlePattern = "^Tornado2 PGM$",
|
||||
TestSceneAllowlist = ["5001", "5006"]
|
||||
};
|
||||
|
||||
Assert.Throws<PlayoutConfigurationException>(
|
||||
() => ValidatedPlayoutOptions.Create(normalTest));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FullFactoryUsesPrivateCapabilityAndNeverReadsAmbientLiveAuthorization()
|
||||
{
|
||||
using var environment = new EnvironmentScope()
|
||||
.Set("MBN_STOCK_PLAYOUT_AUTHORIZE_LIVE_OUTPUT", "false");
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
var events = new ConcurrentQueue<string>();
|
||||
var captureFactory = new FakeEngineFactory(events, new FakeEngine(events));
|
||||
var runner = new PgmCutsSequenceRunner(
|
||||
new FakeSafetyProbe(events),
|
||||
new FakeHashPins(true),
|
||||
captureFactory,
|
||||
new FakeDelay(events),
|
||||
new FakeAssetLeases(true));
|
||||
Assert.True((await runner.ExecuteAsync(Request(scenes.Path))).Completed);
|
||||
var plan = Assert.IsType<PgmCutsSequencePlan>(captureFactory.Plan);
|
||||
var baseline = new PgmDiagnosticTargetSnapshot(
|
||||
true,
|
||||
1,
|
||||
1,
|
||||
true,
|
||||
new PgmDiagnosticTargetIdentity(4_242, 123_456));
|
||||
|
||||
await using var engine = new FullPgmCutsSequenceEngineFactory(
|
||||
new FakeSafetyProbe(new ConcurrentQueue<string>()))
|
||||
.Create(plan, baseline);
|
||||
var authorizationField = typeof(TornadoPlayoutEngine).GetField(
|
||||
"_liveAuthorization",
|
||||
System.Reflection.BindingFlags.Instance |
|
||||
System.Reflection.BindingFlags.NonPublic);
|
||||
Assert.NotNull(authorizationField);
|
||||
var authorization = Assert.IsAssignableFrom<MBN_STOCK_WEBVIEW.Playout.Safety.ILiveAuthorization>(
|
||||
authorizationField.GetValue(engine));
|
||||
|
||||
Assert.True(authorization.IsAuthorizedForThisLaunch);
|
||||
Assert.True(authorization.GetType().IsNestedPrivate);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProcessHashPinRequirementRequiresBothProcessScopedHexPins()
|
||||
{
|
||||
using var environment = new EnvironmentScope()
|
||||
.Set(InstalledK3dInteropMetadata.ApprovedNativeSha256EnvironmentVariable, null)
|
||||
.Set(InstalledK3dInteropMetadata.ApprovedSha256EnvironmentVariable, null);
|
||||
var requirement = new ProcessPgmCutsSequenceHashPinRequirement();
|
||||
|
||||
Assert.False(requirement.IsSatisfied());
|
||||
environment
|
||||
.Set(
|
||||
InstalledK3dInteropMetadata.ApprovedNativeSha256EnvironmentVariable,
|
||||
new string('A', 64))
|
||||
.Set(
|
||||
InstalledK3dInteropMetadata.ApprovedSha256EnvironmentVariable,
|
||||
new string('B', 64));
|
||||
Assert.True(requirement.IsSatisfied());
|
||||
environment.Set(
|
||||
InstalledK3dInteropMetadata.ApprovedSha256EnvironmentVariable,
|
||||
"not-a-hash");
|
||||
Assert.False(requirement.IsSatisfied());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void CutAssetRequirementUsesExactApprovedHashesAndFixedTimeComparison()
|
||||
{
|
||||
Assert.Equal(
|
||||
"99CE3B689A42D8C42BEB09A86FA10C2D7C1AEF4F50D324D81276C1A1E4C4D8A7",
|
||||
FixedPgmCutsSequenceAssetLeaseFactory.FirstApprovedSha256);
|
||||
Assert.Equal(
|
||||
"25CD0AE931F51E4E3B84CE3E6FD21A40DB85464F157A23CC3511D63B336D8757",
|
||||
FixedPgmCutsSequenceAssetLeaseFactory.NextApprovedSha256);
|
||||
|
||||
byte[] content = [1, 3, 3, 7, 42];
|
||||
var contentHash = Convert.ToHexString(
|
||||
System.Security.Cryptography.SHA256.HashData(content));
|
||||
using var matching = new MemoryStream(content, writable: false);
|
||||
using var rejected = new MemoryStream(content, writable: false);
|
||||
|
||||
Assert.True(FixedPgmCutsSequenceAssetLeaseFactory.HasExpectedSha256(
|
||||
matching,
|
||||
contentHash));
|
||||
Assert.False(FixedPgmCutsSequenceAssetLeaseFactory.HasExpectedSha256(
|
||||
rejected,
|
||||
FixedPgmCutsSequenceAssetLeaseFactory.FirstApprovedSha256));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SceneTrustRequiresFixedLocalRootAndFilesUnderThatRoot()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("5001.t2s", "5006.t2s");
|
||||
using var outside = TemporarySceneDirectory.Create("outside.t2s");
|
||||
|
||||
Assert.True(PgmCutsSequenceRunner.IsTrustedLocalScenePaths(
|
||||
scenes.Path,
|
||||
Path.Combine(scenes.Path, "5001.t2s"),
|
||||
Path.Combine(scenes.Path, "5006.t2s")));
|
||||
Assert.False(PgmCutsSequenceRunner.IsTrustedLocalScenePaths(
|
||||
scenes.Path,
|
||||
Path.Combine(scenes.Path, "5001.t2s"),
|
||||
Path.Combine(outside.Path, "outside.t2s")));
|
||||
Assert.False(PgmCutsSequenceRunner.IsTrustedLocalScenePaths(
|
||||
"\\\\server\\share\\Cuts",
|
||||
"\\\\server\\share\\Cuts\\5001.t2s"));
|
||||
}
|
||||
|
||||
private static PgmCutsSequenceRunner Runner(
|
||||
ConcurrentQueue<string> events,
|
||||
FakeEngine engine) => new(
|
||||
new FakeSafetyProbe(events),
|
||||
new FakeHashPins(true),
|
||||
new FakeEngineFactory(events, engine),
|
||||
new FakeDelay(events),
|
||||
new FakeAssetLeases(true));
|
||||
|
||||
private static PgmCutsSequenceRequest Request(string sceneRoot) => new(
|
||||
"127.0.0.1",
|
||||
30_001,
|
||||
"Tornado2 PGM",
|
||||
sceneRoot,
|
||||
5_000);
|
||||
|
||||
private static bool IsOutputCommand(string value) =>
|
||||
value.StartsWith("Prepare:", StringComparison.Ordinal) ||
|
||||
value == "TakeIn" ||
|
||||
value.StartsWith("Next:", StringComparison.Ordinal) ||
|
||||
value.StartsWith("TakeOut:", StringComparison.Ordinal);
|
||||
|
||||
private static string[] EngineCalls(ConcurrentQueue<string> events) =>
|
||||
events.Where(value =>
|
||||
value == "Connect" ||
|
||||
IsOutputCommand(value) ||
|
||||
value is "Disconnect" or "Quarantine" or "Dispose").ToArray();
|
||||
|
||||
private sealed class FakeHashPins : IPgmCutsSequenceHashPinRequirement
|
||||
{
|
||||
private readonly bool _satisfied;
|
||||
|
||||
public FakeHashPins(bool satisfied)
|
||||
{
|
||||
_satisfied = satisfied;
|
||||
}
|
||||
|
||||
public bool IsSatisfied() => _satisfied;
|
||||
}
|
||||
|
||||
private sealed class FakeAssetLeases : IPgmCutsSequenceAssetLeaseFactory
|
||||
{
|
||||
private readonly bool _satisfied;
|
||||
|
||||
public FakeAssetLeases(bool satisfied)
|
||||
{
|
||||
_satisfied = satisfied;
|
||||
}
|
||||
|
||||
public bool TryAcquire(
|
||||
PgmCutsSequencePlan plan,
|
||||
out IPgmCutsSequenceAssetLease? lease)
|
||||
{
|
||||
if (!_satisfied)
|
||||
{
|
||||
lease = null;
|
||||
return false;
|
||||
}
|
||||
|
||||
lease = new FakeAssetLease();
|
||||
return true;
|
||||
}
|
||||
|
||||
private sealed class FakeAssetLease : IPgmCutsSequenceAssetLease
|
||||
{
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class FakeSafetyProbe : IPgmDiagnosticSafetyProbe
|
||||
{
|
||||
private static readonly PgmDiagnosticTargetSnapshot Baseline = new(
|
||||
true,
|
||||
1,
|
||||
1,
|
||||
true,
|
||||
new PgmDiagnosticTargetIdentity(4_242, 123_456));
|
||||
private static readonly PgmDiagnosticTargetSnapshot Changed = new(
|
||||
true,
|
||||
1,
|
||||
1,
|
||||
true,
|
||||
new PgmDiagnosticTargetIdentity(4_243, 123_457));
|
||||
|
||||
private readonly ConcurrentQueue<string> _events;
|
||||
private readonly int? _changedCapture;
|
||||
private int _captureCount;
|
||||
|
||||
public FakeSafetyProbe(
|
||||
ConcurrentQueue<string> events,
|
||||
int? changedCapture = null)
|
||||
{
|
||||
_events = events;
|
||||
_changedCapture = changedCapture;
|
||||
}
|
||||
|
||||
public PgmDiagnosticTargetSnapshot Capture(
|
||||
PgmConnectDiagnosticRequest request,
|
||||
System.Net.IPAddress parsedHost)
|
||||
{
|
||||
Assert.Equal("127.0.0.1", parsedHost.ToString());
|
||||
Assert.Equal(30_001, request.Port);
|
||||
var count = Interlocked.Increment(ref _captureCount);
|
||||
_events.Enqueue($"Safety:{count}");
|
||||
return _changedCapture is { } changed && count >= changed
|
||||
? Changed
|
||||
: Baseline;
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class FakeDelay : IPgmCutsSequenceDelay
|
||||
{
|
||||
private readonly ConcurrentQueue<string> _events;
|
||||
private int _calls;
|
||||
|
||||
public FakeDelay(ConcurrentQueue<string> events)
|
||||
{
|
||||
_events = events;
|
||||
}
|
||||
|
||||
public int? FailureOnCall { get; init; }
|
||||
|
||||
public Task DelayAsync(TimeSpan delay, CancellationToken cancellationToken)
|
||||
{
|
||||
var call = Interlocked.Increment(ref _calls);
|
||||
_events.Enqueue($"Delay:{delay.TotalMilliseconds:0}");
|
||||
if (FailureOnCall == call)
|
||||
{
|
||||
throw new OperationCanceledException("fake observation interruption");
|
||||
}
|
||||
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class FakeEngineFactory : IPgmCutsSequenceEngineFactory
|
||||
{
|
||||
private readonly ConcurrentQueue<string> _events;
|
||||
private readonly FakeEngine _engine;
|
||||
|
||||
public FakeEngineFactory(
|
||||
ConcurrentQueue<string> events,
|
||||
FakeEngine engine)
|
||||
{
|
||||
_events = events;
|
||||
_engine = engine;
|
||||
}
|
||||
|
||||
public int CreateCount { get; private set; }
|
||||
|
||||
public PgmCutsSequencePlan? Plan { get; private set; }
|
||||
|
||||
public IPlayoutEngine Create(
|
||||
PgmCutsSequencePlan plan,
|
||||
PgmDiagnosticTargetSnapshot baseline)
|
||||
{
|
||||
CreateCount++;
|
||||
Plan = plan;
|
||||
Assert.True(baseline.IsEligible);
|
||||
_events.Enqueue("CreateEngine");
|
||||
return _engine;
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class FakeEngine : IPlayoutEngine
|
||||
{
|
||||
private readonly ConcurrentQueue<string> _events;
|
||||
|
||||
public FakeEngine(ConcurrentQueue<string> events)
|
||||
{
|
||||
_events = events;
|
||||
}
|
||||
|
||||
public Dictionary<PgmCutsSequenceStage, PlayoutResultCode> Results { get; } = [];
|
||||
|
||||
public Exception? QuarantineFailure { get; init; }
|
||||
|
||||
public PlayoutStatus Status { get; } = new(
|
||||
PlayoutMode.Live,
|
||||
PlayoutConnectionState.Disconnected,
|
||||
false,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
null,
|
||||
null,
|
||||
"fake",
|
||||
DateTimeOffset.UnixEpoch,
|
||||
0);
|
||||
|
||||
public event EventHandler<PlayoutStatusChangedEventArgs>? StatusChanged
|
||||
{
|
||||
add { }
|
||||
remove { }
|
||||
}
|
||||
|
||||
public Task<PlayoutResult> ConnectAsync(CancellationToken cancellationToken = default)
|
||||
{
|
||||
_events.Enqueue("Connect");
|
||||
return Result(PgmCutsSequenceStage.Connect, PlayoutOperation.Connect);
|
||||
}
|
||||
|
||||
public Task<PlayoutResult> DisconnectAsync(CancellationToken cancellationToken = default)
|
||||
{
|
||||
_events.Enqueue("Disconnect");
|
||||
return Result(PgmCutsSequenceStage.Disconnect, PlayoutOperation.Disconnect);
|
||||
}
|
||||
|
||||
public Task<PlayoutResult> PrepareAsync(
|
||||
PlayoutCue cue,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
_events.Enqueue($"Prepare:{cue.SceneName}");
|
||||
return Result(PgmCutsSequenceStage.PrepareFirst, PlayoutOperation.Prepare);
|
||||
}
|
||||
|
||||
public Task<PlayoutResult> TakeInAsync(CancellationToken cancellationToken = default)
|
||||
{
|
||||
_events.Enqueue("TakeIn");
|
||||
return Result(PgmCutsSequenceStage.TakeIn, PlayoutOperation.TakeIn);
|
||||
}
|
||||
|
||||
public Task<PlayoutResult> NextAsync(
|
||||
PlayoutCue cue,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
_events.Enqueue($"Next:{cue.SceneName}");
|
||||
return Result(PgmCutsSequenceStage.Next, PlayoutOperation.Next);
|
||||
}
|
||||
|
||||
public Task<PlayoutResult> TakeOutAsync(
|
||||
PlayoutTakeOutScope scope,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
_events.Enqueue($"TakeOut:{scope}");
|
||||
return Result(PgmCutsSequenceStage.TakeOut, PlayoutOperation.TakeOut);
|
||||
}
|
||||
|
||||
public ValueTask QuarantineAsync()
|
||||
{
|
||||
_events.Enqueue("Quarantine");
|
||||
return QuarantineFailure is null
|
||||
? ValueTask.CompletedTask
|
||||
: ValueTask.FromException(QuarantineFailure);
|
||||
}
|
||||
|
||||
public ValueTask DisposeAsync()
|
||||
{
|
||||
_events.Enqueue("Dispose");
|
||||
return ValueTask.CompletedTask;
|
||||
}
|
||||
|
||||
private Task<PlayoutResult> Result(
|
||||
PgmCutsSequenceStage stage,
|
||||
PlayoutOperation operation)
|
||||
{
|
||||
var code = Results.GetValueOrDefault(stage, PlayoutResultCode.Success);
|
||||
return Task.FromResult(new PlayoutResult(
|
||||
operation,
|
||||
code,
|
||||
false,
|
||||
"fake",
|
||||
DateTimeOffset.UnixEpoch));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -31,7 +31,19 @@ internal sealed class FakeK3dSessionFactory : IK3dSessionFactory
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class FakeK3dSession : IK3dSession
|
||||
internal sealed class FakeK3dSdkDispatchLatch(bool allow = true) : IK3dSdkDispatchLatch
|
||||
{
|
||||
public IDisposable? TryClaim() => allow ? new Claim() : null;
|
||||
|
||||
private sealed class Claim : IDisposable
|
||||
{
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class FakeK3dSession : IK3dGuardedConnectSession
|
||||
{
|
||||
private int _disposed;
|
||||
private int _ktapDispatchGate;
|
||||
@@ -66,24 +78,53 @@ internal sealed class FakeK3dSession : IK3dSession
|
||||
public Action<string>? CallObserver { get; set; }
|
||||
|
||||
public void Connect(ValidatedPlayoutOptions options)
|
||||
=> ConnectCore(options, finalSafetyCheck: null);
|
||||
|
||||
public void ConnectGuarded(
|
||||
ValidatedPlayoutOptions options,
|
||||
Func<bool> finalSafetyCheck,
|
||||
IK3dSdkDispatchLatch dispatchLatch)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(finalSafetyCheck);
|
||||
ArgumentNullException.ThrowIfNull(dispatchLatch);
|
||||
ConnectCore(options, finalSafetyCheck, dispatchLatch);
|
||||
}
|
||||
|
||||
private void ConnectCore(
|
||||
ValidatedPlayoutOptions options,
|
||||
Func<bool>? finalSafetyCheck,
|
||||
IK3dSdkDispatchLatch? dispatchLatch = null)
|
||||
{
|
||||
Record("Connect");
|
||||
BeforeKtapDispatchAction?.Invoke();
|
||||
if (Interlocked.CompareExchange(ref _ktapDispatchGate, 1, 0) != 0)
|
||||
if (finalSafetyCheck is not null && !finalSafetyCheck())
|
||||
{
|
||||
throw new InvalidOperationException("Fake KTAP dispatch was prevented.");
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
LastKtapConnectState = PlayoutKtapConnectState.Attempted;
|
||||
Interlocked.Increment(ref _ktapDispatchCount);
|
||||
try
|
||||
using (var dispatchClaim = dispatchLatch?.TryClaim())
|
||||
{
|
||||
ConnectAction?.Invoke(options);
|
||||
}
|
||||
catch
|
||||
{
|
||||
LastKtapConnectState = PlayoutKtapConnectState.Failed;
|
||||
throw;
|
||||
if (dispatchLatch is not null && dispatchClaim is null)
|
||||
{
|
||||
throw new K3dConnectSafetyException();
|
||||
}
|
||||
|
||||
if (Interlocked.CompareExchange(ref _ktapDispatchGate, 1, 0) != 0)
|
||||
{
|
||||
throw new InvalidOperationException("Fake KTAP dispatch was prevented.");
|
||||
}
|
||||
|
||||
LastKtapConnectState = PlayoutKtapConnectState.Attempted;
|
||||
Interlocked.Increment(ref _ktapDispatchCount);
|
||||
try
|
||||
{
|
||||
ConnectAction?.Invoke(options);
|
||||
}
|
||||
catch
|
||||
{
|
||||
LastKtapConnectState = PlayoutKtapConnectState.Failed;
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
LastKtapConnectState = PlayoutKtapConnectState.AcceptedUnconfirmed;
|
||||
|
||||
@@ -263,6 +263,79 @@ public sealed class SmokeCommandLineTests
|
||||
Assert.Equal("invalid-port", parsed.ErrorCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Parse_PgmCutsSequenceRequiresItsOwnLongAcknowledgementAndFixedInputs()
|
||||
{
|
||||
var complete = ValidPgmCutsArguments();
|
||||
var missingAcknowledgement = SmokeCommandLine.Parse(
|
||||
complete.Where(value => value != SmokeCommandLine.PgmCutsSequenceAcknowledgementFlag)
|
||||
.ToArray());
|
||||
var wrongAcknowledgement = SmokeCommandLine.Parse(
|
||||
complete.Select(value => value == SmokeCommandLine.PgmCutsSequenceAcknowledgementFlag
|
||||
? SmokeCommandLine.AcknowledgementFlag
|
||||
: value).ToArray());
|
||||
var parsed = SmokeCommandLine.Parse(complete);
|
||||
var explicitObservation = SmokeCommandLine.Parse(
|
||||
complete.Concat(["--observe-ms", "5000"]).ToArray());
|
||||
|
||||
Assert.False(missingAcknowledgement.IsValid);
|
||||
Assert.Equal("missing-acknowledgement", missingAcknowledgement.ErrorCode);
|
||||
Assert.False(wrongAcknowledgement.IsValid);
|
||||
Assert.Equal("unknown-option", wrongAcknowledgement.ErrorCode);
|
||||
Assert.True(parsed.IsValid);
|
||||
Assert.True(explicitObservation.IsValid);
|
||||
Assert.Equal(SmokeCommandKind.PgmCutsSequence, parsed.Invocation.Kind);
|
||||
Assert.Equal(5_000, parsed.Invocation.ObservationMilliseconds);
|
||||
Assert.Equal("C:\\Cuts", parsed.Invocation.SceneRoot);
|
||||
Assert.Null(parsed.Invocation.ConfigPath);
|
||||
Assert.Null(parsed.Invocation.PrepareSceneCode);
|
||||
Assert.Null(parsed.Invocation.NextSceneCode);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("--config")]
|
||||
[InlineData("--prepare")]
|
||||
[InlineData("--next")]
|
||||
public void Parse_PgmCutsSequenceRejectsConfigAndCallerSelectedScenes(string option)
|
||||
{
|
||||
var parsed = SmokeCommandLine.Parse(
|
||||
ValidPgmCutsArguments().Concat([option, "sensitive-value"]).ToArray());
|
||||
|
||||
Assert.False(parsed.IsValid);
|
||||
Assert.Equal("unknown-option", parsed.ErrorCode);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("4999")]
|
||||
[InlineData("5001")]
|
||||
[InlineData("secret")]
|
||||
public void Parse_PgmCutsSequenceOnlyAcceptsExplicitFiveSecondObservation(string value)
|
||||
{
|
||||
var parsed = SmokeCommandLine.Parse(
|
||||
ValidPgmCutsArguments().Concat(["--observe-ms", value]).ToArray());
|
||||
|
||||
Assert.False(parsed.IsValid);
|
||||
Assert.Equal("invalid-observation-duration", parsed.ErrorCode);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("localhost", "C:\\Cuts")]
|
||||
[InlineData("192.0.2.1", "C:\\Cuts")]
|
||||
[InlineData("127.0.0.1", "relative-cuts")]
|
||||
public void Parse_PgmCutsSequenceRequiresLiteralLoopbackAndAbsoluteSceneRoot(
|
||||
string host,
|
||||
string sceneRoot)
|
||||
{
|
||||
var arguments = ValidPgmCutsArguments();
|
||||
arguments[Array.IndexOf(arguments, "127.0.0.1")] = host;
|
||||
arguments[Array.IndexOf(arguments, "C:\\Cuts")] = sceneRoot;
|
||||
|
||||
var parsed = SmokeCommandLine.Parse(arguments);
|
||||
|
||||
Assert.False(parsed.IsValid);
|
||||
Assert.Equal("missing-or-invalid-required-option", parsed.ErrorCode);
|
||||
}
|
||||
|
||||
private static SmokeCommandParseResult ValidSequenceArguments(
|
||||
string prepare = "5001",
|
||||
string observation = "1000") =>
|
||||
@@ -279,4 +352,18 @@ public sealed class SmokeCommandLineTests
|
||||
"--observe-ms",
|
||||
observation
|
||||
]);
|
||||
|
||||
private static string[] ValidPgmCutsArguments() =>
|
||||
[
|
||||
"--pgm-cuts-sequence",
|
||||
SmokeCommandLine.PgmCutsSequenceAcknowledgementFlag,
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--port",
|
||||
"30001",
|
||||
"--expected-pgm-window-title",
|
||||
"Tornado2 PGM",
|
||||
"--scene-root",
|
||||
"C:\\Cuts"
|
||||
];
|
||||
}
|
||||
|
||||
@@ -327,6 +327,640 @@ public sealed class TornadoPlayoutEngineTests
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_FinalTargetFailure_PreventsKtapAndAbandonsWithoutDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var session = new FakeK3dSession();
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => false,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
var result = await engine.ConnectAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.Failed, result.Code);
|
||||
Assert.Equal(0, session.KtapDispatchCount);
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_KtapFailure_IsOutcomeUnknownAndNeverReconnectable()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var session = new FakeK3dSession
|
||||
{
|
||||
ConnectAction = _ => throw new COMException("fake ambiguous KTAP failure")
|
||||
};
|
||||
var sessionFactory = new FakeK3dSessionFactory(() => session);
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
sessionFactory,
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
var result = await engine.ConnectAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, result.Code);
|
||||
Assert.Equal(PlayoutConnectionState.OutcomeUnknown, engine.Status.State);
|
||||
Assert.False(engine.Status.IsCommandAvailable);
|
||||
Assert.Equal(1, session.KtapDispatchCount);
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
await engine.PollProcessOnceAsync(CancellationToken.None);
|
||||
Assert.Equal(1, sessionFactory.CreateCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_CallbackCancellationAfterKtap_IsOutcomeUnknown()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var session = new FakeK3dSession
|
||||
{
|
||||
ConnectAction = _ => throw new OperationCanceledException(
|
||||
"fake cancellation after KTAP dispatch")
|
||||
};
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
var result = await engine.ConnectAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, result.Code);
|
||||
Assert.Equal(PlayoutConnectionState.OutcomeUnknown, engine.Status.State);
|
||||
Assert.Equal(1, session.KtapDispatchCount);
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_CancelledBeforeSerialization_RemainsKnownAndDoesNotCreateSession()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var sessionFactory = new FakeK3dSessionFactory();
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
sessionFactory,
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true);
|
||||
using var cancellation = new CancellationTokenSource();
|
||||
cancellation.Cancel();
|
||||
|
||||
var result = await engine.ConnectAsync(cancellation.Token);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.Cancelled, result.Code);
|
||||
Assert.Equal(0, sessionFactory.CreateCount);
|
||||
Assert.Equal(PlayoutKtapConnectState.NotAttempted, engine.Status.LastKtapConnectState);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_SlowSafetyCheckPastTimeoutNeverDispatchesKtap()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
using var safetyStarted = new ManualResetEventSlim();
|
||||
using var releaseSafety = new ManualResetEventSlim();
|
||||
var session = new FakeK3dSession();
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () =>
|
||||
{
|
||||
safetyStarted.Set();
|
||||
Assert.True(releaseSafety.Wait(TimeSpan.FromSeconds(5)));
|
||||
return true;
|
||||
},
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
try
|
||||
{
|
||||
var connectTask = engine.ConnectAsync(CancellationToken.None);
|
||||
Assert.True(safetyStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var result = await connectTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, result.Code);
|
||||
Assert.Equal(0, session.KtapDispatchCount);
|
||||
releaseSafety.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
Assert.Equal(0, session.KtapDispatchCount);
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releaseSafety.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_TimeoutWinningAtomicClaimPreventsKtapDispatch()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
using var claimStarted = new ManualResetEventSlim();
|
||||
using var releaseClaim = new ManualResetEventSlim();
|
||||
var session = new FakeK3dSession();
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true,
|
||||
beforeSdkDispatchClaim: () =>
|
||||
{
|
||||
claimStarted.Set();
|
||||
Assert.True(releaseClaim.Wait(TimeSpan.FromSeconds(5)));
|
||||
});
|
||||
|
||||
try
|
||||
{
|
||||
var connectTask = engine.ConnectAsync(CancellationToken.None);
|
||||
Assert.True(claimStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var connect = await connectTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, connect.Code);
|
||||
Assert.Equal(0, session.KtapDispatchCount);
|
||||
releaseClaim.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
Assert.Equal(0, session.KtapDispatchCount);
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releaseClaim.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedConnect_PostGenerationChange_AbandonsWithoutDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var original = ProcessSnapshot("pgm-generation-a");
|
||||
var changed = ProcessSnapshot("pgm-generation-b");
|
||||
var session = new FakeK3dSession();
|
||||
var finalChecks = 0;
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(original, changed),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => Interlocked.Increment(ref finalChecks) == 1,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
var result = await engine.ConnectAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, result.Code);
|
||||
Assert.Equal(1, finalChecks);
|
||||
Assert.Equal(1, session.KtapDispatchCount);
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_PreCommandGenerationChange_AbandonsWithoutDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var original = ProcessSnapshot("pgm-generation-a");
|
||||
var changed = ProcessSnapshot("pgm-generation-b");
|
||||
var session = new FakeK3dSession();
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(original, original, changed),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var prepare = await engine.PrepareAsync(
|
||||
Cue("test-scene"),
|
||||
CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, prepare.Code);
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
Assert.DoesNotContain(session.Calls, call =>
|
||||
call.StartsWith("Prepare:", StringComparison.Ordinal));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_AmbiguousPrepareFailure_AbandonsWithoutDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var stable = ProcessSnapshot("pgm-generation-a");
|
||||
var session = new FakeK3dSession
|
||||
{
|
||||
PrepareAction = (_, _) => throw new COMException("fake ambiguous prepare failure")
|
||||
};
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(stable),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var prepare = await engine.PrepareAsync(
|
||||
Cue("test-scene"),
|
||||
CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, prepare.Code);
|
||||
Assert.Equal(
|
||||
new[] { "Connect", "Prepare:test-scene", "Abandon" },
|
||||
session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_NextChecksTargetBetweenPrepareAndPlay()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create(
|
||||
"test-scene.t2s",
|
||||
"next-scene.t2s");
|
||||
var stable = ProcessSnapshot("pgm-generation-a");
|
||||
var session = new FakeK3dSession();
|
||||
var safetyChecks = 0;
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene", "next-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(stable),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () =>
|
||||
Interlocked.Increment(ref safetyChecks) != 7,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
Assert.True((await engine.PrepareAsync(
|
||||
Cue("test-scene"),
|
||||
CancellationToken.None)).IsSuccess);
|
||||
Assert.True((await engine.TakeInAsync(CancellationToken.None)).IsSuccess);
|
||||
|
||||
var next = await engine.NextAsync(
|
||||
Cue("next-scene"),
|
||||
CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, next.Code);
|
||||
Assert.Equal(7, safetyChecks);
|
||||
Assert.Equal(1, session.Calls.Count(call => call == "Play"));
|
||||
Assert.Equal(
|
||||
new[]
|
||||
{
|
||||
"Connect",
|
||||
"Prepare:test-scene",
|
||||
"Play",
|
||||
"Prepare:next-scene",
|
||||
"Abandon"
|
||||
},
|
||||
session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_DisconnectPrecheckFailureAbandonsWithoutDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var stable = ProcessSnapshot("pgm-generation-a");
|
||||
var session = new FakeK3dSession();
|
||||
var safetyChecks = 0;
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(stable),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () =>
|
||||
Interlocked.Increment(ref safetyChecks) == 1,
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var disconnect = await engine.DisconnectAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, disconnect.Code);
|
||||
Assert.Equal(2, safetyChecks);
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_SlowCommandSafetyCheckPastTimeoutNeverDispatchesSdkCall()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
using var safetyStarted = new ManualResetEventSlim();
|
||||
using var releaseSafety = new ManualResetEventSlim();
|
||||
var stable = ProcessSnapshot("pgm-generation-a");
|
||||
var session = new FakeK3dSession();
|
||||
var safetyChecks = 0;
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(stable),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () =>
|
||||
{
|
||||
if (Interlocked.Increment(ref safetyChecks) == 2)
|
||||
{
|
||||
safetyStarted.Set();
|
||||
Assert.True(releaseSafety.Wait(TimeSpan.FromSeconds(5)));
|
||||
}
|
||||
|
||||
return true;
|
||||
},
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
try
|
||||
{
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var prepareTask = engine.PrepareAsync(
|
||||
Cue("test-scene"),
|
||||
CancellationToken.None);
|
||||
Assert.True(safetyStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var prepare = await prepareTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, prepare.Code);
|
||||
releaseSafety.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
Assert.DoesNotContain(
|
||||
session.Calls,
|
||||
call => call.StartsWith("Prepare:", StringComparison.Ordinal));
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releaseSafety.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_SlowDisconnectSafetyCheckPastTimeoutNeverDisconnects()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
using var safetyStarted = new ManualResetEventSlim();
|
||||
using var releaseSafety = new ManualResetEventSlim();
|
||||
var stable = ProcessSnapshot("pgm-generation-a");
|
||||
var session = new FakeK3dSession();
|
||||
var safetyChecks = 0;
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(stable),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () =>
|
||||
{
|
||||
if (Interlocked.Increment(ref safetyChecks) == 2)
|
||||
{
|
||||
safetyStarted.Set();
|
||||
Assert.True(releaseSafety.Wait(TimeSpan.FromSeconds(5)));
|
||||
}
|
||||
|
||||
return true;
|
||||
},
|
||||
abandonOnTargetMismatch: true);
|
||||
|
||||
try
|
||||
{
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var disconnectTask = engine.DisconnectAsync(CancellationToken.None);
|
||||
Assert.True(safetyStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var disconnect = await disconnectTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, disconnect.Code);
|
||||
releaseSafety.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releaseSafety.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_TimeoutWinningAtomicCommandClaimPreventsPrepare()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
using var claimStarted = new ManualResetEventSlim();
|
||||
using var releaseClaim = new ManualResetEventSlim();
|
||||
var session = new FakeK3dSession();
|
||||
var claims = 0;
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(ProcessSnapshot("pgm-generation-a")),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true,
|
||||
beforeSdkDispatchClaim: () =>
|
||||
{
|
||||
if (Interlocked.Increment(ref claims) == 2)
|
||||
{
|
||||
claimStarted.Set();
|
||||
Assert.True(releaseClaim.Wait(TimeSpan.FromSeconds(5)));
|
||||
}
|
||||
});
|
||||
|
||||
try
|
||||
{
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var prepareTask = engine.PrepareAsync(
|
||||
Cue("test-scene"),
|
||||
CancellationToken.None);
|
||||
Assert.True(claimStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var prepare = await prepareTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, prepare.Code);
|
||||
releaseClaim.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
Assert.DoesNotContain(
|
||||
session.Calls,
|
||||
call => call.StartsWith("Prepare:", StringComparison.Ordinal));
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releaseClaim.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_NextUsesSeparateAtomicClaimsForPrepareAndPlay()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create(
|
||||
"test-scene.t2s",
|
||||
"next-scene.t2s");
|
||||
using var playClaimStarted = new ManualResetEventSlim();
|
||||
using var releasePlayClaim = new ManualResetEventSlim();
|
||||
var session = new FakeK3dSession();
|
||||
var claims = 0;
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene", "next-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(ProcessSnapshot("pgm-generation-a")),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true,
|
||||
beforeSdkDispatchClaim: () =>
|
||||
{
|
||||
if (Interlocked.Increment(ref claims) == 5)
|
||||
{
|
||||
playClaimStarted.Set();
|
||||
Assert.True(releasePlayClaim.Wait(TimeSpan.FromSeconds(5)));
|
||||
}
|
||||
});
|
||||
|
||||
try
|
||||
{
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
Assert.True((await engine.PrepareAsync(
|
||||
Cue("test-scene"),
|
||||
CancellationToken.None)).IsSuccess);
|
||||
Assert.True((await engine.TakeInAsync(CancellationToken.None)).IsSuccess);
|
||||
var nextTask = engine.NextAsync(
|
||||
Cue("next-scene"),
|
||||
CancellationToken.None);
|
||||
Assert.True(playClaimStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var next = await nextTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, next.Code);
|
||||
Assert.Contains("Prepare:next-scene", session.Calls);
|
||||
Assert.Equal(1, session.Calls.Count(call => call == "Play"));
|
||||
releasePlayClaim.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
Assert.Equal(1, session.Calls.Count(call => call == "Play"));
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releasePlayClaim.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task GuardedTargetPolicy_TimeoutWinningAtomicDisconnectClaimPreventsDisconnect()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
using var claimStarted = new ManualResetEventSlim();
|
||||
using var releaseClaim = new ManualResetEventSlim();
|
||||
var session = new FakeK3dSession();
|
||||
var claims = 0;
|
||||
var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
new FakeK3dSessionFactory(() => session),
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(ProcessSnapshot("pgm-generation-a")),
|
||||
liveAuthorized: false,
|
||||
finalConnectSafetyCheck: () => true,
|
||||
abandonOnTargetMismatch: true,
|
||||
beforeSdkDispatchClaim: () =>
|
||||
{
|
||||
if (Interlocked.Increment(ref claims) == 2)
|
||||
{
|
||||
claimStarted.Set();
|
||||
Assert.True(releaseClaim.Wait(TimeSpan.FromSeconds(5)));
|
||||
}
|
||||
});
|
||||
|
||||
try
|
||||
{
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
var disconnectTask = engine.DisconnectAsync(CancellationToken.None);
|
||||
Assert.True(claimStarted.Wait(TimeSpan.FromSeconds(2)));
|
||||
|
||||
var disconnect = await disconnectTask;
|
||||
|
||||
Assert.Equal(PlayoutResultCode.OutcomeUnknown, disconnect.Code);
|
||||
releaseClaim.Set();
|
||||
Assert.True(SpinWait.SpinUntil(
|
||||
() => session.Calls.Contains("Abandon"),
|
||||
TimeSpan.FromSeconds(2)));
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
}
|
||||
finally
|
||||
{
|
||||
releaseClaim.Set();
|
||||
await engine.DisposeAsync();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task LastKtapEvidence_SurvivesLaterFailureBeforeAnotherDispatch()
|
||||
{
|
||||
@@ -1171,6 +1805,34 @@ public sealed class TornadoPlayoutEngineTests
|
||||
Assert.Equal(1, sessionFactory.CreateCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Quarantine_WhenLocalAbandonFails_ReportsFailureAndNeverDisconnectsOrDisposes()
|
||||
{
|
||||
using var scenes = TemporarySceneDirectory.Create("test-scene.t2s");
|
||||
var session = new FakeK3dSession
|
||||
{
|
||||
AbandonAction = () => throw new InvalidOperationException(
|
||||
"fake local abandon failure")
|
||||
};
|
||||
var sessionFactory = new FakeK3dSessionFactory(() => session);
|
||||
await using var engine = CreateEngine(
|
||||
TestOptions(scenes.Path, "test-scene"),
|
||||
sessionFactory,
|
||||
new FakeRegistrationProbe(),
|
||||
new FakeTornadoProcessProbe(new TornadoProcessSnapshot(1, 1, 0)),
|
||||
liveAuthorized: false);
|
||||
|
||||
Assert.True((await engine.ConnectAsync(CancellationToken.None)).IsSuccess);
|
||||
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(
|
||||
() => engine.QuarantineAsync().AsTask());
|
||||
|
||||
Assert.Equal(new[] { "Connect", "Abandon" }, session.Calls);
|
||||
Assert.DoesNotContain("Disconnect", session.Calls);
|
||||
Assert.DoesNotContain("Dispose", session.Calls);
|
||||
Assert.Equal(1, sessionFactory.CreateCount);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
@@ -1219,7 +1881,10 @@ public sealed class TornadoPlayoutEngineTests
|
||||
FakeK3dSessionFactory sessionFactory,
|
||||
FakeRegistrationProbe registrationProbe,
|
||||
FakeTornadoProcessProbe processProbe,
|
||||
bool liveAuthorized)
|
||||
bool liveAuthorized,
|
||||
Func<bool>? finalConnectSafetyCheck = null,
|
||||
bool abandonOnTargetMismatch = false,
|
||||
Action? beforeSdkDispatchClaim = null)
|
||||
{
|
||||
var options = ValidatedPlayoutOptions.Create(rawOptions);
|
||||
IStaDispatcher? dispatcher = options.Mode is PlayoutMode.Test or PlayoutMode.Live
|
||||
@@ -1233,7 +1898,10 @@ public sealed class TornadoPlayoutEngineTests
|
||||
dispatcher,
|
||||
new FakeLiveAuthorization(liveAuthorized),
|
||||
TimeProvider.System,
|
||||
startMonitor: false);
|
||||
startMonitor: false,
|
||||
finalConnectSafetyCheck: finalConnectSafetyCheck,
|
||||
abandonOnTargetMismatch: abandonOnTargetMismatch,
|
||||
beforeSdkDispatchClaim: beforeSdkDispatchClaim);
|
||||
}
|
||||
|
||||
private static PlayoutOptions TestOptions(
|
||||
|
||||
@@ -29,6 +29,8 @@ return parsed.Invocation.Kind switch
|
||||
await RunIsolatedTestCommandAsync(parsed.Invocation, cancellation.Token),
|
||||
SmokeCommandKind.PgmConnectDiagnostic =>
|
||||
await RunPgmConnectDiagnosticAsync(parsed.Invocation, cancellation.Token),
|
||||
SmokeCommandKind.PgmCutsSequence =>
|
||||
await RunPgmCutsSequenceAsync(parsed.Invocation, cancellation.Token),
|
||||
_ => Usage("unknown-command")
|
||||
};
|
||||
|
||||
@@ -286,6 +288,75 @@ static async Task<int> RunPgmConnectDiagnosticAsync(
|
||||
return report.OutcomeUnknown ? 5 : 4;
|
||||
}
|
||||
|
||||
static async Task<int> RunPgmCutsSequenceAsync(
|
||||
SmokeCommandInvocation invocation,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (IsolatedTestCommandPreflight.HasPlayoutEnvironmentOverride(
|
||||
Environment.GetEnvironmentVariables()))
|
||||
{
|
||||
WritePgmCutsSequenceFailure("environment-override-present");
|
||||
return 65;
|
||||
}
|
||||
|
||||
var request = new PgmCutsSequenceRequest(
|
||||
invocation.Host ?? string.Empty,
|
||||
invocation.Port ?? 0,
|
||||
invocation.ExpectedPgmWindowTitle ?? string.Empty,
|
||||
invocation.SceneRoot ?? string.Empty,
|
||||
invocation.ObservationMilliseconds ?? 0);
|
||||
PgmCutsSequenceReport report;
|
||||
try
|
||||
{
|
||||
report = await new PgmCutsSequenceRunner()
|
||||
.ExecuteAsync(
|
||||
request,
|
||||
stage =>
|
||||
{
|
||||
WriteProgressJson(new
|
||||
{
|
||||
@event = "observation-start",
|
||||
stage = ToObservationWireValue(stage)
|
||||
});
|
||||
},
|
||||
cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
catch
|
||||
{
|
||||
WritePgmCutsSequenceFailure("internal-sequence-failure");
|
||||
return 70;
|
||||
}
|
||||
|
||||
WriteJson(new
|
||||
{
|
||||
command = "pgm-cuts-sequence",
|
||||
report.RequestValidated,
|
||||
report.HashPinsPresent,
|
||||
report.TargetGatePassed,
|
||||
report.EngineCreated,
|
||||
report.RenderCommandAttempted,
|
||||
report.DisconnectAttempted,
|
||||
report.QuarantineAttempted,
|
||||
report.QuarantineCompleted,
|
||||
report.Completed,
|
||||
report.OutcomeUnknown,
|
||||
commands = report.Commands.Select(command => new
|
||||
{
|
||||
stage = ToSequenceStageWireValue(command.Stage),
|
||||
code = command.Code.ToString()
|
||||
}),
|
||||
report.ErrorCode
|
||||
});
|
||||
|
||||
if (report.Completed && !report.OutcomeUnknown)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
return report.OutcomeUnknown ? 5 : 4;
|
||||
}
|
||||
|
||||
static void WritePgmDiagnosticFailure(string errorCode) =>
|
||||
WriteJson(new
|
||||
{
|
||||
@@ -313,6 +384,24 @@ static void WritePgmDiagnosticFailure(string errorCode) =>
|
||||
errorCode
|
||||
});
|
||||
|
||||
static void WritePgmCutsSequenceFailure(string errorCode) =>
|
||||
WriteJson(new
|
||||
{
|
||||
command = "pgm-cuts-sequence",
|
||||
requestValidated = false,
|
||||
hashPinsPresent = false,
|
||||
targetGatePassed = false,
|
||||
engineCreated = false,
|
||||
renderCommandAttempted = false,
|
||||
disconnectAttempted = false,
|
||||
quarantineAttempted = false,
|
||||
quarantineCompleted = false,
|
||||
completed = false,
|
||||
outcomeUnknown = false,
|
||||
commands = Array.Empty<object>(),
|
||||
errorCode
|
||||
});
|
||||
|
||||
static string ToWireValue(PlayoutKtapConnectState state) => state switch
|
||||
{
|
||||
PlayoutKtapConnectState.NotAttempted => "not-attempted",
|
||||
@@ -322,6 +411,25 @@ static string ToWireValue(PlayoutKtapConnectState state) => state switch
|
||||
_ => "not-attempted"
|
||||
};
|
||||
|
||||
static string ToObservationWireValue(PgmCutsObservationStage stage) => stage switch
|
||||
{
|
||||
PgmCutsObservationStage.First => "first",
|
||||
PgmCutsObservationStage.Next => "next",
|
||||
PgmCutsObservationStage.TakeOut => "take-out",
|
||||
_ => "unknown"
|
||||
};
|
||||
|
||||
static string ToSequenceStageWireValue(PgmCutsSequenceStage stage) => stage switch
|
||||
{
|
||||
PgmCutsSequenceStage.Connect => "connect",
|
||||
PgmCutsSequenceStage.PrepareFirst => "prepare-first",
|
||||
PgmCutsSequenceStage.TakeIn => "take-in",
|
||||
PgmCutsSequenceStage.Next => "next",
|
||||
PgmCutsSequenceStage.TakeOut => "take-out",
|
||||
PgmCutsSequenceStage.Disconnect => "disconnect",
|
||||
_ => "unknown"
|
||||
};
|
||||
|
||||
static void WritePreflightFailure(string command, string errorCode) =>
|
||||
WriteJson(new
|
||||
{
|
||||
@@ -349,6 +457,15 @@ static void WriteJson<T>(T value) =>
|
||||
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
|
||||
}));
|
||||
|
||||
static void WriteProgressJson<T>(T value)
|
||||
{
|
||||
Console.WriteLine(JsonSerializer.Serialize(value, new JsonSerializerOptions
|
||||
{
|
||||
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
|
||||
}));
|
||||
Console.Out.Flush();
|
||||
}
|
||||
|
||||
static int Usage(string? errorCode)
|
||||
{
|
||||
WriteJson(new
|
||||
@@ -366,7 +483,7 @@ static int Usage(string? errorCode)
|
||||
errorCode = errorCode ?? "invalid-command-line"
|
||||
});
|
||||
Console.Error.WriteLine(
|
||||
"Usage: MBN_STOCK_WEBVIEW.PlayoutSmoke [--probe|--dry-run|--test-plan|--test-connect|--test-sequence|--pgm-connect-diagnostic]");
|
||||
"Usage: MBN_STOCK_WEBVIEW.PlayoutSmoke [--probe|--dry-run|--test-plan|--test-connect|--test-sequence|--pgm-connect-diagnostic|--pgm-cuts-sequence]");
|
||||
Console.Error.WriteLine(
|
||||
$"Test commands require {SmokeCommandLine.AcknowledgementFlag} and --config <absolute-path>.");
|
||||
Console.Error.WriteLine(
|
||||
@@ -375,5 +492,7 @@ static int Usage(string? errorCode)
|
||||
"--test-connect and --test-sequence may request render-capable COM after all Test safety gates pass.");
|
||||
Console.Error.WriteLine(
|
||||
$"--pgm-connect-diagnostic requires {SmokeCommandLine.PgmConnectAcknowledgementFlag}, --host, --port and --expected-pgm-window-title; it exposes no render command.");
|
||||
Console.Error.WriteLine(
|
||||
$"--pgm-cuts-sequence requires {SmokeCommandLine.PgmCutsSequenceAcknowledgementFlag}, --host, --port, --expected-pgm-window-title and --scene-root; --observe-ms may only be 5000.");
|
||||
return 64;
|
||||
}
|
||||
|
||||
@@ -8,7 +8,8 @@ internal enum SmokeCommandKind
|
||||
TestPlan,
|
||||
TestConnect,
|
||||
TestSequence,
|
||||
PgmConnectDiagnostic
|
||||
PgmConnectDiagnostic,
|
||||
PgmCutsSequence
|
||||
}
|
||||
|
||||
internal sealed record SmokeCommandInvocation(
|
||||
@@ -19,7 +20,8 @@ internal sealed record SmokeCommandInvocation(
|
||||
int? ObservationMilliseconds = null,
|
||||
string? Host = null,
|
||||
int? Port = null,
|
||||
string? ExpectedPgmWindowTitle = null);
|
||||
string? ExpectedPgmWindowTitle = null,
|
||||
string? SceneRoot = null);
|
||||
|
||||
internal sealed record SmokeCommandParseResult(
|
||||
bool IsValid,
|
||||
@@ -32,6 +34,8 @@ internal static class SmokeCommandLine
|
||||
"--acknowledge-isolated-non-program-test-output";
|
||||
internal const string PgmConnectAcknowledgementFlag =
|
||||
"--i-understand-this-will-contact-current-pgm-tornado-via-ktap-connect-and-disconnect-only";
|
||||
internal const string PgmCutsSequenceAcknowledgementFlag =
|
||||
"--i-understand-this-will-render-current-pgm-tornado-cuts-5001-then-5006-and-take-them-out";
|
||||
|
||||
public static SmokeCommandParseResult Parse(IReadOnlyList<string> arguments)
|
||||
{
|
||||
@@ -57,6 +61,7 @@ internal static class SmokeCommandLine
|
||||
"--test-connect" => SmokeCommandKind.TestConnect,
|
||||
"--test-sequence" => SmokeCommandKind.TestSequence,
|
||||
"--pgm-connect-diagnostic" => SmokeCommandKind.PgmConnectDiagnostic,
|
||||
"--pgm-cuts-sequence" => SmokeCommandKind.PgmCutsSequence,
|
||||
_ => SmokeCommandKind.Invalid
|
||||
};
|
||||
if (kind == SmokeCommandKind.Invalid)
|
||||
@@ -71,6 +76,7 @@ internal static class SmokeCommandLine
|
||||
string? host = null;
|
||||
int? port = null;
|
||||
string? expectedPgmWindowTitle = null;
|
||||
string? sceneRoot = null;
|
||||
var acknowledged = false;
|
||||
|
||||
for (var index = 1; index < arguments.Count; index++)
|
||||
@@ -78,8 +84,10 @@ internal static class SmokeCommandLine
|
||||
var argument = arguments[index];
|
||||
switch (argument)
|
||||
{
|
||||
case AcknowledgementFlag when kind != SmokeCommandKind.PgmConnectDiagnostic:
|
||||
case AcknowledgementFlag when kind is not (
|
||||
SmokeCommandKind.PgmConnectDiagnostic or SmokeCommandKind.PgmCutsSequence):
|
||||
case PgmConnectAcknowledgementFlag when kind == SmokeCommandKind.PgmConnectDiagnostic:
|
||||
case PgmCutsSequenceAcknowledgementFlag when kind == SmokeCommandKind.PgmCutsSequence:
|
||||
if (acknowledged)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
@@ -88,7 +96,8 @@ internal static class SmokeCommandLine
|
||||
acknowledged = true;
|
||||
break;
|
||||
|
||||
case "--host" when kind == SmokeCommandKind.PgmConnectDiagnostic:
|
||||
case "--host" when kind is
|
||||
SmokeCommandKind.PgmConnectDiagnostic or SmokeCommandKind.PgmCutsSequence:
|
||||
if (host is not null)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
@@ -101,7 +110,8 @@ internal static class SmokeCommandLine
|
||||
|
||||
break;
|
||||
|
||||
case "--port" when kind == SmokeCommandKind.PgmConnectDiagnostic:
|
||||
case "--port" when kind is
|
||||
SmokeCommandKind.PgmConnectDiagnostic or SmokeCommandKind.PgmCutsSequence:
|
||||
if (port is not null)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
@@ -122,7 +132,8 @@ internal static class SmokeCommandLine
|
||||
break;
|
||||
|
||||
case "--expected-pgm-window-title"
|
||||
when kind == SmokeCommandKind.PgmConnectDiagnostic:
|
||||
when kind is
|
||||
SmokeCommandKind.PgmConnectDiagnostic or SmokeCommandKind.PgmCutsSequence:
|
||||
if (expectedPgmWindowTitle is not null)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
@@ -135,7 +146,21 @@ internal static class SmokeCommandLine
|
||||
|
||||
break;
|
||||
|
||||
case "--config" when kind != SmokeCommandKind.PgmConnectDiagnostic:
|
||||
case "--scene-root" when kind == SmokeCommandKind.PgmCutsSequence:
|
||||
if (sceneRoot is not null)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
}
|
||||
|
||||
if (!TryReadValue(arguments, ref index, out sceneRoot))
|
||||
{
|
||||
return Invalid("missing-option-value");
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "--config" when kind is not (
|
||||
SmokeCommandKind.PgmConnectDiagnostic or SmokeCommandKind.PgmCutsSequence):
|
||||
if (configPath is not null)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
@@ -194,6 +219,26 @@ internal static class SmokeCommandLine
|
||||
observationMilliseconds = observationValue;
|
||||
break;
|
||||
|
||||
case "--observe-ms" when kind == SmokeCommandKind.PgmCutsSequence:
|
||||
if (observationMilliseconds is not null)
|
||||
{
|
||||
return Invalid("duplicate-option");
|
||||
}
|
||||
|
||||
if (!TryReadValue(arguments, ref index, out var pgmObservationText) ||
|
||||
!int.TryParse(
|
||||
pgmObservationText,
|
||||
System.Globalization.NumberStyles.None,
|
||||
System.Globalization.CultureInfo.InvariantCulture,
|
||||
out var pgmObservationValue) ||
|
||||
pgmObservationValue != 5_000)
|
||||
{
|
||||
return Invalid("invalid-observation-duration");
|
||||
}
|
||||
|
||||
observationMilliseconds = pgmObservationValue;
|
||||
break;
|
||||
|
||||
default:
|
||||
return Invalid("unknown-option");
|
||||
}
|
||||
@@ -220,6 +265,28 @@ internal static class SmokeCommandLine
|
||||
ExpectedPgmWindowTitle: expectedPgmWindowTitle));
|
||||
}
|
||||
|
||||
if (kind == SmokeCommandKind.PgmCutsSequence)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(host) ||
|
||||
!System.Net.IPAddress.TryParse(host, out var literalHost) ||
|
||||
!System.Net.IPAddress.IsLoopback(literalHost) ||
|
||||
port is null ||
|
||||
string.IsNullOrWhiteSpace(expectedPgmWindowTitle) ||
|
||||
string.IsNullOrWhiteSpace(sceneRoot) ||
|
||||
!Path.IsPathFullyQualified(sceneRoot))
|
||||
{
|
||||
return Invalid("missing-or-invalid-required-option");
|
||||
}
|
||||
|
||||
return Valid(new SmokeCommandInvocation(
|
||||
kind,
|
||||
ObservationMilliseconds: observationMilliseconds ?? 5_000,
|
||||
Host: host,
|
||||
Port: port,
|
||||
ExpectedPgmWindowTitle: expectedPgmWindowTitle,
|
||||
SceneRoot: sceneRoot));
|
||||
}
|
||||
|
||||
if (string.IsNullOrWhiteSpace(configPath))
|
||||
{
|
||||
return Invalid("missing-required-option");
|
||||
@@ -254,7 +321,8 @@ internal static class SmokeCommandLine
|
||||
observationMilliseconds,
|
||||
host,
|
||||
port,
|
||||
expectedPgmWindowTitle));
|
||||
expectedPgmWindowTitle,
|
||||
sceneRoot));
|
||||
}
|
||||
|
||||
internal static bool IsSafeSceneCode(string value)
|
||||
|
||||
Reference in New Issue
Block a user