ED! THIS ONE IS EASY: A Supported MFCD/Image Export API!!

Helios, iControl, and other are showing that there is GREAT interest in external instruments for ALL of your products, yet MFCD export is lacking. If ED trotted out a DCS-specific api where a process can connect and grab some 256x256 or 512x512 frames, we (as in, simpit builders) could then make an external programs/devices to connect and get the data without taking an FPS hit :-)

The layout of the API would be simple - just connect to a socket, and specify what image data you want (left/right mfcd, rwr, etc). (The lua export is already most way there.) And then you keep receiving frame/image data until the connection is dropped.

To make it *extra* easy, I'll tell you how to implement it in DirectX9! :thumbup:

I'm assuming that the MFDs are implemented as a surface render-to-texture operation. You can copy the surface from the GPU back into system memory using one of the methods: D3DXSaveSurfaceToFileInMemory or the IDirect3DDevice9::GetRenderTargetData methods would do nicely (there may be other ways to pull it off.) Either way, you end up with a system memory structure that can now be exported via api/network transfer.

This could be VERY easy to perform every other frame, if bandwidth becomes an issue (which I doubt, we're talking about 192k for 256x256, or 768k for 512x512, raw RGB with no compression - nobody needs the alpha) data export could be staggered to capture/export just half the data on alternate frames.

Even if it's just the graphical instruments, this would be a very simple, yet HUGE improvement that will reap lots of community support/addons for sim-pit builders of all types. :pilotfly:

Helios, iControl, and other are showing that there is GREAT interest in external instruments for ALL of your products, yet MFCD export is lacking. If ED trotted out a DCS-specific api where a process can connect and grab some 256x256 or 512x512 frames, we (as in, simpit builders) could then make an external programs/devices to connect and get the data without taking an FPS hit :-)

 

Nice idea. If there were some documented TCP/IP interface, I would allready written a program myself to export those displays on external monitors. There are 8'' USB TFTs out there which are great for displaying ABRISS, SHKVAL or MFCD displays. I'm currently planing to integrate two TM Cougar MFD with such TFTs (and lots of buttons and switches) in a case for that.

But unfortunately I dont think it will be implemented soon (or ever?). So we will have to use the windowed mode. For myself the fps impact is not that great, but the configuration in the lua is a mess.

How do you expect this to reduce FPS impact?

To get the framedata you need to render it somewhere. Either your export target will have to basically run DCS itself, or the PC will have to render whatever's going on there and send the finished frame to your device. The former is not likely, meaning that the latter is required, and you'll thus always have the "problem" anyhow.

That said, graphics programming is WAAAAAY outside of my field so I may very likely be missing something. :P

How do you expect this to reduce FPS impact?

 

Easy (but naive) answer: You could use the fullscreen mode (FSM) with this feature (and use another application to draw the MFCDs on smaller displays). Fullscreen mode renders faster on most graphic cards so you will gain a few fps. But the problem (thats the reason why the answer is a bit naive) might be, that in FSM the graphics card could not render the images for the MFCDs, so it hat to be done by software.

In one or two years all of us have better graphic cards, better cpus and even on my current machine (K2600, GTX 560 Ti) the fps impact for using two small USB TFTs and a second monitor (running A10/BS2 at 4000x1080 at the moment), is not that big. Even with mirros on its playable. Conclusion: Time will solve the issues with fps.

I'm currently only testing not flying for real with 3 and in a few days 4 monitors. Its still playable in window mode at this resolution - and normaly I'm very picky with fps rates.

How do you expect this to reduce FPS impact?

It's *already* rendered once, for the in-cockpit MFD. Instead of rendering it again, simply retrieve the finished texture/surface *from* the GPU back into system memory.

I don't anticipate this being picked up by ED ...so, in my spare time, I started writing my own plugin to make the proper DirectX calls during these past three weeks or so. :smartass:

It's a system tray application should work in both x64 and x86, and could support others plugins, also!) It's going to take a bit, as I've been dolling it up to be very pretty and easy to use. But the end result should be that you can use ANYTHING as an MFCD. Getting it working w/BS2 should be a breeze, once complete.

I'll film and post a demo here once it's more functional. It's nearly functional, I've managed to get access to (realtime) renderdata - I'm currently hunting for the rendersurfaces. So far, the frame rate impact is exactly zero (unless I start logging those calls to disk). And that's on a triple-monitor eyefinity.... in fullscreen. It's my goal to maintain that speed.

Uneducated guess:

you only have to know the address of a shader surface and you can grab this data from the back-buffer of the V-ram .

And almost no Frame-drop.

@zarthrag

Do you have a project put together that is capable to do it already on a rudimentary basis that you could share for testing?

@PeterP - I've been working on it pretty hard on this, polishing as I go along. I should be extracting images by tonight or so, I'll throw up a youtube video once it's alive. I'm purposefully leaving room to expand/add plugins for "additional capabilities" :thumbup:

It's *already* rendered once, for the in-cockpit MFD. Instead of rendering it again, simply retrieve the finished texture/surface *from* the GPU back into system memory.

What happens if you move your head inside the pit so that the MFCDs are out of sight?

I finally got a REALLY rudimentary proof of concept going (video tonight, after some more experimentation.

Unfortunately, the render is done in multiple steps, *AND* requires a shader. First, a RTT operation snaps the MAV/TGP camera. To create the gimbal effect, it's RTT'd into a new texture, via rotation. Followed by some shader voodoo*, and then another texture layered on top for the MFD interface*.

*Denotes speculation

At the moment, I'm reading/writing files to disk. My playback app (written in about 15 minutes) simply watches for a new file and displays it. Enough allow me to surf through textures. I need to stop and build the real transport, and a real playback client. So it'll be a while yet.

@Sobek - Confirmed that they still render, instead of being optimized/occluded away...Whew!

I'm usually not the person that starts to pull someone's leg, but

:)some progress?:)

lol, I've been wrapped up in other things this past few week! But I'm back on it this week. Currently working on the network communication and remote display client.

Thanks for quick reply ! :)

I fully understand - all good things need time .

Oh, and the capture plugin is 100% working already, (minus the transport piece). Tested against both x86 and x64 on DCS:A-10C in full-screen. Full framerate. (proof upcoming, once I get the transport done.)

Oh, and the capture plugin is 100% working already, (minus the transport piece). Tested against both x86 and x64 on DCS:A-10C in full-screen. Full framerate. (proof upcoming, once I get the transport done.)

If you need any help, I already made such a tool (though for BMS) using the exact same principle :). http://www.benchmarksims.org/forum/showthread.php?10677-Beta-Release-GPT-(cockpit-texture-extraction-remote-cockpit-control-shm-mirror).

I recommend spawning a slave thread inside the game process which can handle the data compression and transmission. (I send displays as jpeg frames over network, but ideally you'd want a dedicated video stream.)

If anyone is curious about performance: On my system I cannot measure a performance loss at all anymore with the latest version, though others on other systems have reported as much as 40% fps loss (This can be due to lack of asynchronous texture download support in the gpu drivers, lack of free CPU core for frame encoding or similar issues).

The key is to use double (or triple) buffering (extra textures) in video ram that you download from these copies to system ram (you NEVER lock and download the actual texture that the game is drawing to), and you enqueue the download operation at intelligent times, so that it is performed in the background asynchronously over the course of a whole game render cycle. Then when you have the full frame downloaded to system ram, you encode it using an extra cpu thread so that cpu/gpu cost for the game is absolutely minimal.

This way I can download, encode, and transmit frames 1200x1200 at 50 Hz over TCP, and like I said on my system I cannot consistently even measure any performance loss.

Yoda, thanks for your input!

... things can get shape now ....

:)

(I was thinking to PM you .. now you showed up! :))

Might be irrelevant but has anyone tried http://www.maxivista.com/ and is able to explain if this turns GPU into CPU and bandwidth in the same way?

I have to be honest, I thought i understood techie stuff but this puts me to shame. I've been interested in cheap ways of using multiscreen.

Maxivista, which i don't endorse worked for me, I used a laptop to render a third screen for the main PC. I believe it installs a fake video display though on the main PC. I don't know how much rendering was done on the main PC if any though.

The concept is awesome if you have rubbish old PC's lying around or laptops and want to use them as a part of your main PC screen layout.

PS its not software KVM which some folk get confused about.

Maxvista is OK for Desktop/office work - but it doesn't take any workload from your system - it only higher it. As your system has still to render it and than compress the data and send it to another PC...

When you are looking for a cheap way to expand the capabilities of your GPU-card to attach more monitors via a second low-range GPU - have a look at SoftTH - much faster than adding a USB monitor. - and a much lower power consumption than a second pc.

>>>SoftTH NEWS

You still don't know what SoftTH is? :

 

SoftTH is donation-ware and let you do what the big GPU company's don't allow you to do or only at high costs.

SoftTh let you render every monitor combination at High FPS in full-screen mode. You can even Mix SoftTH with AMD-Eyefinity and/or TH2GO,NVIDIA-Surround !

Especially interesting for DCS. -SoftTH let you use full-screen Mode If you export MDFS/Abris ect. on additional screens . You are not forced to use Windowed mode any more.

Even Helios is working with DCS in Fullscreen on all secondary heads.

And Yes , it will gain your FPS!

 

Some interesting examples what you can do/how it can look:

Any chance we could see the source code for the dll hook part?

I would very much like to make a 64bit compatible injection dll, but since I don't know assembly I've been stuck with microsoft detours as a solution(replacing the d3dcreate function), but it doesn't have free support for 64bit(it cost 10 k$...).

- BUMP -

Come on you guru's whats happening in this space??

Yoda, does leavu work on FC3??

- BUMP -

 

Come on you guru's whats happening in this space??

 

Yoda, does leavu work on FC3??

+1 this would be great to have!

I've been working on a USB MFD solution that goes along w/this project. (There will also be a helios-compatible solution, but it isn't my highest priority atm.) I was hopeful to obtain some 5" square displays, but they are....pricey. Also, there's some rework necessary to get things working properly w/DCS world. I plan to make the hooking source available, once the hardware is finished. Not sure on the MFD specifics though, as it isn't yet complete - as I've got at least one other project that may go along w/this. (Hint: it's an interface device that will be a companion to the oculus rift.)

I hope to see progress in this soon :)

I've just arrived at this thread via a link another member gave me. I have no idea what you guys are talking about but I can tell when I'm being hornswaggled. Mr. Zarthrag seems to be full of ballon juice. I could be wrong and will humbly admit it with proof to the contrary.

John

Balloon juice, indeed!

While I managed to get an image export going. I decided to give-up on the custom MFDs: due to their expense and complexity. The Oculus Rift devkit I got opened up a whole new world when it comes to interface development. The Rift provides (a potential for) situational awareness (once it's HD, at least). But input is far from being solved. I decided to undertake an approach that DID NOT involve 3 monitors, MFDs, direct3d hooking, and custom panels...just to replicate exactly ONE cockpit....

I digress. Here's a (long overdue) update of what I've been working on this past year.

(You should be able to see where I'm trying to go with this.)

December of last year, I made those gloves as a proof-of-concept (literally done over the course of two weeks). I've since trashed that design and reworked them from a model that uses flex-sensors, to one that uses accelerometers. Then, I reworked that to include gyroscopes, as well. My latest prototype even has pcb that I should get back in 2 weeks or so. Once it's soldered and working, I'll be offering a limited number of dev units.

I've already developed a basic API that provides a skeletal model (as shown in the demo from the first prototype). It's meant to be uses along with STEM or kinect to obtain the body's pose, and the gloves simply provide further skeletal information. Simple as that.

The second prototype (only wired a few of the fingers as a functional test, breakout pcbs made it way too bulky to possibly wear)

[ATTACH]90196[/ATTACH]

I'm willing to provide a set if someone is interested, and can prove that they are capable of creating an interface into the virtual cockpit - preferably with a skeletal hand, rendered in-cockpit. (This especially includes you Eagle Dynamics guys!)

And, if you're (still) skeptical (which I can understand), just wait until you see the production model in operation ;-)

Additionally, I've bundled up the code for the MFD solution I had. It's a year old, and untested with DCS world, but it did successfully hook DCS automatically under both windows XP and Win7 (32 and 64bit). It could technically (and was intended to) hook ANY game via a profiler that watches for game startup (also fully-functional!)

There's likely some code-rot going on in there, but here's a snap of the code, as well as it's git repo, for those interested. (Given my pursuit of VR, I see no need to keep this under-wraps.)

https://www.dropbox.com/s/jt0uj0qq5cw1djt/SimpitX.zip