Remote desktop lessons: hidden cursors, black frames, and testing Windows without a screen

I build an RMM called Moorfox, and this week I shipped four small features for its remote desktop: sessions now start view-only, the “someone is connected” banner on the user’s machine can be dragged out of the way, the operator can see the end user’s mouse pointer, and the screen picker on multi-monitor machines now actually picks the screen. None of these is a big feature. Almost everything I learned came from getting them working and then proving they worked on a Windows box in EC2 that I could not see or touch. Notes worth writing down, mostly for future me.

The cursor is never in the frame

If you capture a screen with DXGI Desktop Duplication on Windows or an X11 root-window grab on Linux, the mouse pointer is not in the pixels. Both APIs exclude it by design, and for a screenshot tool that is what you want. For a support session it is a problem, because the person on the other end keeps saying “this thing here” while pointing at something you cannot see.

We already had a WebRTC data channel carrying the operator’s input to the agent, with mouse coordinates normalised to the frame. The fix was the mirror image: the agent polls the pointer position once per captured frame (GetCursorPos on Windows against the captured output’s desktop origin, XQueryPointer on X11 minus the crop offset), and sends it back on its own data channel only when it moved. The viewer draws a small pointer with a “user” tag on top of the video. It costs about a kilobyte a second while the mouse is moving and nothing when it is still, and it turns “click the… no, left of that” conversations into “click where I’m pointing”.

A click-through window cannot be dragged

The session banner is a native layered Win32 window, top-centre, always on top, so the person at the machine always knows someone is connected. It was also created with WS_EX_TRANSPARENT, which makes it click-through, and it sat exactly where a maximised browser puts its address bar.

Making it draggable took two changes and one trap. Drop WS_EX_TRANSPARENT so the window receives mouse input at all, then answer WM_NCHITTEST with HTCAPTION so the whole surface behaves like a title bar and DefWindowProc runs the whole drag loop for you. No drag code, no capture handling. The trap was the repaint path: the banner redraws itself on DPI and monitor changes, and redrawing recomputed the top-centre placement, so the window would snap back over the address bar the user had dragged it away from. You have to remember that the user moved it and repaint in place.

The option that was plumbed all the way through and connected to nothing

The multi-monitor picker already existed. The display ID travelled from the browser through the REST API, the signalling hub, the agent’s WebSocket, across a process boundary into the session helper, into the capture config… and the last function ignored it and opened the primary display. Every layer in between looked finished, which is exactly why nobody noticed.

The actual selection work was small. Windows display IDs from EnumDisplayDevices resolve to GDI device names like \\.\DISPLAY2, DXGI outputs carry the same names in their descriptions, and you have to create the D3D device on the adapter that owns the output or DuplicateOutput fails on multi-GPU machines. On Linux the X root window is one big surface across monitors, so picking a screen means cropping the grab to that output’s xrandr geometry. What I will remember is the shape of the bug, though: when an option crosses six layers, trace it into the final consumer before calling any of the layers done.

The first run on a different machine found a real bug in minutes

My rewritten DXGI setup enumerated adapters and outputs through a DXGI factory and took adapter 0, output 0 as the default. Worked on every machine I developed against. First session on the EC2 box failed instantly: its first adapter has no outputs at all, and interestingly the factory listed no outputs anywhere, while the old code path - let D3D11CreateDevice pick the default adapter, take its first output - kept working fine on the same machine.

So the default stayed on the boring proven path, and only an explicit screen choice goes through enumeration. And when a choice cannot be honoured (the monitor got unplugged since the list was fetched, the ID is stale), the agent logs it and falls back to the primary screen instead of refusing the session. Someone asking for help does not care which monitor comes up first; they care that the session opens.

Testing Windows GUI code with no screen attached

The test box was Windows Server on EC2. No RDP client in this workflow, no monitor, and the interesting behaviour is all visual. Three tricks made it workable:

1. SSM gets you a shell, but in session 0. Anything you run through AWS Systems Manager lands in the services session, which has no desktop worth capturing and cannot inject input into the user’s session. To touch the real desktop, register a scheduled task with an Interactive logon principal for the console user and start it. That is how the drag test ran.

2. aws ec2 get-console-screenshot is your eyes. It returns a JPEG of the actual framebuffer, logon screen included. It lags a few seconds, so screenshot well inside the window of the state you care about - I burned one run by screenshotting at second 300 of a 300-second session and catching the banner a moment after it closed.

3. Input injection is a P/Invoke one-liner. SetCursorPos plus mouse_event in a PowerShell scheduled task, stepping the cursor twenty times between press and release, dragged the banner like a human would. Console screenshots before and after showed it top-centre, then parked where the drag left it, and it stayed put. That pair of images was the whole acceptance test.

Two EC2 traps that cost me an afternoon

I tried to fake a second monitor with the Amyuni usbmmidd virtual display driver so I could test multi-screen capture. On Windows Server it broke DXGI duplication of the console outright - every session failed with 0x887a0002 until I removed the device and rebooted. As far as I can tell there is no way to simulate a second capturable output on an EC2 box, so true two-monitor capture goes on the physical-hardware test list.

The second trap was worse. On EC2’s emulated display adapter, Desktop Duplication succeeds and delivers frames at 30 fps, and every frame is black. GDI’s CopyFromScreen in the same session sees the real desktop. My headless receiver had been reporting healthy framerates and bitrates the whole time, because it counts bytes, and bytes were flowing. I only caught it by drawing the received video into a canvas and measuring brightness: zero. Throughput tests and content tests are different tests, and on VMs without a real GPU you may need GDI capture as a fallback if you want sessions to show anything.

Is this session P2P or going through the relay?

WebRTC tries a direct connection first (STUN helps both sides discover their public addresses and punch through NAT), and falls back to TURN, a relay both sides can reach, when the NATs are too hostile. The relay carries the full video bitrate but only ever sees DTLS-SRTP ciphertext. Most people deploy coturn for this; Moorfox runs its own small relay on pion’s TURN library with the same short-lived HMAC credential scheme, because it shares code and deployment with the rest of the control plane.

You can always answer “is this session relayed?” from chrome://webrtc-internals by finding the selected candidate pair and checking for a relay candidate type. But the viewer already polls getStats every second for the fps counter, and the same report contains the selected pair. Fifteen lines later the footer says “P2P” or “relay” next to the round-trip time. If a question matters enough that you keep opening a debug page for it, the product should answer it.

Next on the list from this round: a proper two-monitor pass on physical hardware, and deciding whether a GDI fallback for GPU-less VMs is worth carrying. If you are building on Desktop Duplication, test on a cheap VM early - the failure modes are nothing like the ones on your desk.