Jump to content

A DCS:W Tailored Alternative to VAC for Better Voice Control


Recommended Posts

VAC is a great piece of software and I've spend a considerable amount of time creating profiles and tweaking the sim to make the user experience even better when using VAC (See VAC Friendly Radio Mod in DCS Mods forum). Unfortunately, using VAC to issue radio commands in DCS remains clumsy due to the context sensitive nature of many commands and I am beginning to wonder if my efforts would be better directed toward creating a tailored voice recognition utility that interacts directly with the sim, bypassing the radio communication menu entirely.

 

I think it is unrealistic to expect ED to develop voice recognition capabilities in addition to their current projects so the concept for a third party application to fill the void is roughly as follows: Through the sim's export interface, the proposed utility maintains a list of named radio entities in the simulation world (ATC, JTAC, AWACS etc.) on the tuned frequency (or more simply parralel the sim's radio communication menu). When a voice command is issued, the utility looks to see if that entity exists and sends the appropriate command directly to the sim using the input interface rather than through keypress macros activating the communications menu.

 

I've found through putting together the VAC Friendly Radio Mod that the functions required to get the list of radio entities are exposed in the speech lua files. A means of calling these and returning their output to the external utility would satisfy the first requirement for such a program. The second requirement is the ability to issue the radio commands directly with the appropriate parameters. This may prove to be more challenging (unless a developer can share some details of how the system works) but I beleive that it is still technically possible from what I've been reading in the forums. If this is not possible, but the first requirement can be satisfied, then it will still be possible to issue the commands using keypress macros.

 

The ultimate goals is to be able to tune the appropriate radio frequency, issue a voice command such as "Darkstar, Hawg 1-1 available for tasking, playtime 0+45" and have the proposed utility find and address the command to the correct entity (the JTAC named "Darkstar") within the simulation. This will make the interface between user and sim more intuitive and provide a more immersive experience benefitting both single-player and mutli-player users.

 

I'm hoping some of the more experienced and technically competent modders, as well as any developers who happen upon this post, will join the discussion here and weigh in on whether something like this is techinally possible before I invest a great deal of time pursuing it. This thread is also intended to serve as brainstorming session and to feel out if anyone else would be interested in working on this project as well.

 

Thanks in advance for everyone's input.

Link to comment
Share on other sites

I've looked into issuing commands directly and couldn't come up with a good way to do it.

 

If anyone else knows of a way, I'd really like to know as well. I'd love to incorporate it into iControl DCS.

iControl DCS/DCS Virtual Cockpit - Full featured iPad Cockpit - Now with Android support!

A10 Virtual Cockpit Free - Free limited functionality version of iControl DCS!

DCS Virtual Cockpit - Android version!

 

Follow on Twitter for all the latest news

Link to comment
Share on other sites

I've recently done a custom recognizer in C#, it's open source, and was very easy:

 

http://zarthcode.com/products/eespeak-a-voice-based-lookup-tool/

 

It has a limited grammar, and is very accurate. If you have set commands, I imagine it would be easy to implement something similar to VAC, but customized for DCS.

 

Personally, I use VoiceAttack to play the new Wing Commander, it's MUCH simpler to use.

 

The real challenge is designing/creating a great grammar. You want it to be easy to remember/use, difficult for the recognizer to misunderstand. The choices need to work well, and friendly to running alongside teamspeak (if listening all of the time), or require a button to activate/listen. (It helps to allow the user to customize all of this).

 

If you start work on such a project, I'll gladly help wherever I can.

Link to comment
Share on other sites

Not possible through the export Lua environment. You need to be accessing the "main simulation" aka "server" environment for this.

 

I'm not sure how to select the various radio menu items via Lua, or even if it is possible. I've never studied or been much interested in VAC or modding the radio menus.

Intelligent discourse can only begin with the honest admission of your own fallibility.

Member of the Virtual Tactical Air Group: http://vtacticalairgroup.com/

Lua scripts and mods:

MIssion Scripting Tools (Mist): http://forums.eagle.ru/showthread.php?t=98616

Slmod version 7.0 for DCS: World: http://forums.eagle.ru/showthread.php?t=80979

Now includes remote server administration tools for kicking, banning, loading missions, etc.

Link to comment
Share on other sites

AJ.eightFive I am using your VAC work and I thank you for that because you have saved me countless hours of work.

I am concerned as we run multi-player server and due to cheat prevention had to disable exports. Unfortunately DCS server does not allow to scale integrity checks in such way that it allows multiple selection of "approved mods" while not allowing to connect with any other user changes to .lua files. So far VAC method is 100% compatible with integrity checks since its method is not connected with .lua exports and works both for FC2 and DCS modules.

51PVO Founding member (DEC2007-)

100KIAP Founding member (DEC2018-)

 

:: Shaman aka [100☭] Shamansky

tail# 44 or 444

[sIGPIC][/sIGPIC] 100KIAP Regiment Early Warning & Control officer

Link to comment
Share on other sites

  • 2 weeks later...

Hi AJ,

 

This is indeed a very interesting project!

 

Since Zarthrag has already discussed about the grammar design challenge, I’m just going to discuss about the technical part of this project.

 

Several years ago, I’ve used Windows speech recognition to speak to the tower and my wingmen in Lock-On. Windows Vista/7 natively support voice recognition, it can press keys and launch programs at your order. If you are running XP, you can download an optional (free) module from Microsoft web site. (Ask Google for more details about the integrated voice recognition: http://www.microsoft.com/enable/products/windowsvista/speech.aspx)

 

There is something even more interesting: Windows also offers a COM API to receive voice commands from the user in your C++ program (for example). You can read this excellent tutorial on code project: http://www.codeproject.com/Articles/11771/Voice-Command

 

This means that under Windows, you don’t have to code yourself any voice recognition module or even use an external program. You just have to use the native API from Microsoft with the Microsoft or a third party voice recognition driver. This solves the first technical problem.

 

Now that we have a clean way to receive voice commands from the user, we have to think about the best way to inject them into DCS.

 

From what I know about Export.lua there is no easy way to give orders to a specific unit in the battlefield (I’m ignoring the player aircraft). It is possible that Combined Arms will add such kind of feature in the future.

 

You may also want to the check the editor to see if the new script functionalities can enable some sort of communications between the battlefield objects and an external script like export.lua.

 

In Combined Arms, can you command ground units with the “radio”?

tacview-signature-512x128x24.png
Link to comment
Share on other sites

  • 4 years later...

This is a bit of a necro, but I was recently working with radio commands with renewed interest due to getting the Oculus Rift HMD for testing with DCS and other games. Please point me to more recent or more relevant threads on the subject if you know of any, but I was left wondering how the voice command system could be improved.

 

People have been putting huge effort into profiles for popular voice command software to make sense of the complicated DCS radio command structure. I can only admire their work, but I really think people shouldn't need to go to such lengths to get voice commands to work. I'm not really a programmer so perhaps you guys can tell me if my idea has any kind of merit whatsoever.

 

Step 1:

ED add another Lua file to give modders access to radio commands. Basically anytime the radio menu is opened and DCS waits for a key press it would also activate this Lua file, provide an array of radio commands ["Flight", "Two", "Three", ...] and expect an array index in return (the option chosen by the user). My gut feeling is that this would mean a fairly reasonable amount of effort from the ED programmers - but of course I could be wrong.

 

Step 2:

Individual modders or open source communities can now utilize this interface to connect to for example the speech recognition engine that comes with Windows. This would probably involve quite a bit of effort, but since the real heavy lifting is already taken care of by MicroSoft, it should be doable. The modders have already done such amazing things so I don't think this kind of a mod would be too much to ask, especially if a group of people work together.

 

Basically the way I imagine it works (I know nothing about the internal working of the speech recognition libraries) is that at some point you can actually send an array of expected word or sentences and tell it to figure out if any of them can be recognized from the mic input. This could then be used to create a totally dynamic system that would work regardless of what kind of radio options DCS decides to use in the future. It would simply send the options as plain text to the Lua interface and the external software could listen to those exact sentences and then tell DCS which one the user selected (if any).

 

As a fallback option you could have "option one", "option two" and so on - this is actually what I chose to do with my recent VAC profile. Not very immersive, but foolproof and since VAC gets to work with a very limited set of commands it does a pretty good job of recognizing the correct options even though my English is pretty bad.

 

Eventually (given suitable licensing) when the mod gets good enough ED could just pick it up and implement it as a part of the default DCS package.

 

Again, am I missing something really important or do you think this could work? Of course many people seem to want seamless commands (like "Two, engage air defences with missiles" all in one go) instead of going step by step through the menus, which would require a bit more complicated interface with the simulator, but at least I would be very happy if I could just read the one command at a time and then progress to the next menu level and so on.

 

I think voice commands for radio ops is just such a useful feature - especially with HMD devices when your access to the keyboard is quite limited - that it would catch on pretty well if there was a seamlessly working, simple implementation available. I would appreciate it if you let me know what you think of the idea or point me at more recent discussions on the subject if you know of any.

My blog full of incoherent ramblings on random subjects: https://anttiilomaki.wordpress.com/

Link to comment
Share on other sites

  • 4 weeks later...

I'm looking into some options for a possible next-gen version of VAICOM 2.0:

 

https://forums.eagle.ru/showthread.php?p=2534446#post2534446

 

The idea would be to replace the sending of keystrokes sequences with talking to DCS API's directly, injecting voice commands to AI recipients such as ATC, wingmen, AWACS, etc.

 

I'm thinking along the lines of developing a VoiceAttack plugin dll, I checked out the basics on that, Yoda has come up with DCS remote 2 which provides a REST API on which commands can be injected over network socket / HTTP into DCS at any time during a mission

 

https://forums.eagle.ru/showthread.php?t=162282

 

From first impressions this looks like a powerful tool and I got DCS to recognize basic Lo commands in export context.

 

The missing piece at the moment is that comms menu commands cannot (?) be injected using the export environment.

 

The ATC.lua has been modified by Wartsell for static ATC menu tables:

 

https://forums.eagle.ru/showthread.php?t=138994

 

this script and others like comms.lua appear to give clues on involved objects e.g. from lines such as

 

command = sendMessage.new(Message.wMsgLeaderRequestAzimuth)

 

..making me guess that there exist (global?) message tables that can be utilized.

 

The question is how, and investigations on my end continue.

However any hints from experienced DCS hackers reading this would definitely be welcome to get me going in the right direction :pilotfly:

 

Thanks a lot

Hollywood

There is no spoon.

 

Avatar_old_80x80.gif.0c105925ce4b9f5b87697ea37cbde317.gif

VAICOM PRO plugin for DCS World

www.vaicompro.com

Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...