VRK - a virtual reality enabled kneeboard with touch and ink support - ED Forums
 


Notices

Reply
 
Thread Tools Display Modes
Old 08-05-2019, 12:56 PM   #1
AMVI_Rider
Junior Member
 
AMVI_Rider's Avatar
 
Join Date: Jun 2012
Location: Italy
Posts: 96
Default VRK - a virtual reality enabled kneeboard with touch and ink support

What is VRK?
Given the latest improvements of VR hardware, the most limiting factor for VR flight simulation in VR is the inability to access the documents and taking notes.
So, the name says it all: VRK or VR Kneeboard is a kneeboard application for flight simulators. It designed to be used in the VR environment where accessing paper and taking notes is annoying to impossible.

How does it work?
The application generates an overlay in the VR world that can be placed and sized at will. When focused (looked at) the kneeboard can change its transparency and size to increase readability.
The kneeboard is divided in tabs that can be created at will. Each tab can contain either:
  • An imported PDF (useful for charts, frequency tables, briefings, etc.).
  • An empty Notes tab, single or multiple pages per tab is supported.
PDF and Notes can be used with a pen (digitizer or pen tablet) to take notes either on existing documents or on the bank sheet.
Scrolling and zooming are controlled by touch swipe/pinch, gestures or physical tablet buttons.

What about the immersion?
After some months of use, the best configuration to use VRK is the standalone mode with a pen tablet.
In the end, the lack of touch support on the pen tablets is compensated by the presence of the tablet buttons.

The original client/server mode will be supported for users with proper tablets and the ones willing to test the application.

Supported tablets are:
Wacom (all Intuos line is expected to work in WinTab mode)
Huion (using the custom driver, see Appendix A of the Quick start guide provided with the installer)

Reported working models:
Wacom: Intuos S, Intuos M (both BT and USB version)
Huion: HS64, H640P, H950P

XP-Pen devices are not working with VRK.

What else?
Here are some plans for this tool for the future:
  • Use the tool during my squadron activities
  • Fix bugs!
  • Support for native OculusVR APIs (under study)
  • Android support: I know that affordable Windows tablets w/ pen are like unicorns
  • Support for VR controllers: attaching and controlling the panel with them

Supported platforms
The tool has been developed using OpenVR (SteamVR) which, to my limited understanding of the VR APIs is the only platform supporting external overlays into applications “out of the box”.
The good news for Oculus owners (like me) is that SteamVR is complatible with Oculus devices too. It is just required that your (flight) simulator is compatible with SteamVR and can be forced to SteamVR mode.
In case of DCS this is done by adding a command line option ( --force_steam_VR ) or by using Skatezilla DCS Updater.

Both Pimax 5k and HTC Vive have been tested. All the other models of the family are expected to work as well.

Where to download the tool?
Like DCSMP and the other tools I designed for flight simulation, the installer is hosted by my squadron website:
VRK Installer

Any comment and suggestion is appreciated, just consider that I write code and support the tool in my very limited spare time.

Example videos:




Changelog (relevant items only)
[1.1.3.18]
[NEW] DCS: hooks to detect mission loading and aircraft type
[NEW] Support for keyboard and joystick/HID control
[NEW] Brightness control

[1.1.3.14]
[FIX] Mousewheel not interacting with zoom/pan anymore
[FIX] Default Alpha changing Default Width
[CHG] Supersampling revisited to improve image quality at low zoom
[CHG] VRK window will not stay topmost anymore
[CHG] Config window will not automatically reset seated position (button added)
[NEW] Bookmarks
[NEW] Configurable tabs and bookmarks location
[NEW] Config Apply function
[NEW] Load directory and DCS miz files
[NEW] Automatic load of DCS multiplayer session Kneeboard (preliminar)
[NEW] Automatic restore of last layout (save on close)

[1.1.3.11]
[NEW] Custom driver for Huion pen tablets
[NEW] Zoom Out function
[NEW] Configurable delay in transition from focused to un-focused mode
[NEW] Clickable page change buttons

[1.1.3.9]
[CHG] Removed buttons functions for switching Note Mode
[FIX] Windows width calculation should not happen in maximized mode
[FIX] Crash on startup if SteamVR is installed but no HMD connected
[NEW] WinTab support for tablet not complying with relative buttons mode
[CHG] Dynamic cursor size switching
[NEW] UcLogic (Huion, XP-Pen) custom driver
[NEW] VR on/off functions
[NEW] VR sizing mode switch function

Known bugs and limitations ( 1.1.3.18 ):
  • Kneeboard is not visible from the bottom side. Pay attention to rotations when re-positioning the thing, if the kneeboard disappears, put back the default values.
  • XP-Pen devices are not supported by Custom Driver
  • DCS: Automatic selection of aircraft does not work in single player mode
Attached Thumbnails
Click image for larger version

Name:	VRK-GUI1.png
Views:	444
Size:	52.8 KB
ID:	214965   Click image for larger version

Name:	VRK-GUI2.png
Views:	464
Size:	52.0 KB
ID:	214966   Click image for larger version

Name:	VRK-GUI3.png
Views:	440
Size:	52.8 KB
ID:	214967  

Click image for larger version

Name:	VRK-VR1.jpg
Views:	860
Size:	162.8 KB
ID:	214974   Click image for larger version

Name:	VRK-Kneeboard.jpg
Views:	726
Size:	104.9 KB
ID:	214975  
__________________

Author of DCSMP and VRK
Spoiler:
Ryzen 5 3600X - 32GB DD4 3200C14 Win10 64 - Geforce GTX 1080Ti
Hotas Warhog + Virpil T-50 base - Saitek Combat Rudder Pedals - Cougar MFCDs - Custom head tracker
35" UWQHD main display + 22" MFCD/Helios display / Rift S
2x256 GB SSD - 2Tb Caviar Green

Last edited by AMVI_Rider; 10-24-2019 at 11:25 AM.
AMVI_Rider is offline   Reply With Quote
Old 08-05-2019, 01:34 PM   #2
OttoPus
Member
 
Join Date: Aug 2012
Location: Sardinia, Italy
Posts: 523
Default

Outstanding idea, keep up the good work!
OttoPus is offline   Reply With Quote
Old 08-06-2019, 01:34 PM   #3
Habu_69
Member
 
Join Date: Jul 2011
Location: USA VA
Posts: 854
Default

Potentially a great tool for us VR pilots. Easy to install. But how about some more detailed instructions on client/server connection. I assume one installs the program on both devices and chooses the relevant client or server mode. Then what? Wired connection or wireless? What net address? Port? What do all those SETTINGS parameters do?
Habu_69 is offline   Reply With Quote
Old 08-06-2019, 01:41 PM   #4
EntropySG
Member
 
EntropySG's Avatar
 
Join Date: Feb 2015
Posts: 739
Default

outstanding idea, will try it as soon as I get home.
the link you provided seems to be broken for me, this one it is I assume

https://www.amvi.it/download429-vr-kneeboard.html
__________________
EntropySG is offline   Reply With Quote
Old 08-06-2019, 04:37 PM   #5
AMVI_Rider
Junior Member
 
AMVI_Rider's Avatar
 
Join Date: Jun 2012
Location: Italy
Posts: 96
Default

Quote:
Originally Posted by Habu_69 View Post
Potentially a great tool for us VR pilots. Easy to install. But how about some more detailed instructions on client/server connection. I assume one installs the program on both devices and chooses the relevant client or server mode. Then what? Wired connection or wireless? What net address? Port? What do all those SETTINGS parameters do?
The client/server mode uses you home network (LAN). It is supposed that both are connected to your "home" network via cable and/or WiFi.

I will be back with some instructions very soon, maybe with a video if I manage to get the overlay recorded too. Frankly I'm thinking of completely changing the network setup part with some kind of automatic discovery of the two machines (for many good reasons).

For the connection, the only parameter you have to change is the network address field on the CLIENT machine.
In this field you have to put the LAN IP address (a.k.a internal network address) of the SERVER machine. If you don't know how to obtain it follow these instructions:

https://www.digitalcitizen.life/find-ip-address-windows

These address are usually in a form like 192.168.x.y (usually x = 0 and y is an "arbitrary" number from 1 to 253).

Once you put this address in the "Network Address" field of the CLIENT configuration, click OK.
Don't change the port unless you absolutely know what you are doing. In any case, put the same number in both CLIENT and SERVER.
Everything else is fine for your first test.



Once the configuration window closes, the CLIENT tries to establish the connection with the SERVER.

As you will see, when the connection goes up the big red cross in the server application (which indicates that the connection is down) will go away. Also, the CLIENT title bar will change showing the "CONNECTED" indication.
Re-connection in case of connection loss is automatic.



Meaning of the other parameters
Virtual Reality checkbox: enables the VR rendering on the helmet (if availabe). You have to set it on on the SERVER. Setting it off on the CLIENT is not required unless you have a visor installed on the client too.
Yaw, Pith, Roll, X, Y, Z is the pose (position+orientation) of the kneeboard in the virtual world. The zero is your head. Default values is approximately on top of your right leg.
Default/Focused Width is the width of the kneeboard in the VR. Default is when you are not looking directly at it. The application will switch to the Focused value when you are directly looking at the kneeboard.
Default/Focused Alpha is the opacity of the kneeboard with 0 meaning completely transparent and 1 completely opaque. Also in this case the two values are switched automatically when you look at the kneeboard.
Focus delay It is not used at the moment. In the future it will let you change the delay between the switch from Default and Focused representation.

When I say "look at the kneeboard" I mean that the kneeboard is in front of your eyes.


Quote:
Originally Posted by EntropySG View Post
outstanding idea, will try it as soon as I get home.
the link you provided seems to be broken for me, this one it is I assume

https://www.amvi.it/download429-vr-kneeboard.html
Thank you, yes the URL is that one.
Hope to have some feedback from you guys! The application lacks of some polishing and maturity because I'm focused more on the behavior and use cases, time will bring all the little things.
Attached Thumbnails
Click image for larger version

Name:	VRK-ClientConfig.png
Views:	3213
Size:	32.4 KB
ID:	215037   Click image for larger version

Name:	VRK-ServerDisconnected.png
Views:	3179
Size:	7.5 KB
ID:	215038  
__________________

Author of DCSMP and VRK
Spoiler:
Ryzen 5 3600X - 32GB DD4 3200C14 Win10 64 - Geforce GTX 1080Ti
Hotas Warhog + Virpil T-50 base - Saitek Combat Rudder Pedals - Cougar MFCDs - Custom head tracker
35" UWQHD main display + 22" MFCD/Helios display / Rift S
2x256 GB SSD - 2Tb Caviar Green
AMVI_Rider is offline   Reply With Quote
Old 08-06-2019, 05:39 PM   #6
EntropySG
Member
 
EntropySG's Avatar
 
Join Date: Feb 2015
Posts: 739
Default

tried standalone, cant seem to interact with it. using mousewheel or clicks but it wont flip pages. installing on my surface pro and see if it works in server/client mode with touch and ink
__________________
EntropySG is offline   Reply With Quote
Old 08-06-2019, 08:35 PM   #7
lucky-hendrix
Member
 
Join Date: Nov 2006
Posts: 344
Default

This looks amazing !!!! Can't wait to try it !

Sent from my VTR-L09 using Tapatalk
lucky-hendrix is offline   Reply With Quote
Old 08-06-2019, 08:54 PM   #8
lucky-hendrix
Member
 
Join Date: Nov 2006
Posts: 344
Default

Is the development 'open source' ? Is there anyway we can help ?

Sent from my VTR-L09 using Tapatalk
lucky-hendrix is offline   Reply With Quote
Old 08-06-2019, 09:56 PM   #9
AMVI_Rider
Junior Member
 
AMVI_Rider's Avatar
 
Join Date: Jun 2012
Location: Italy
Posts: 96
Default

Quote:
Originally Posted by lucky-hendrix View Post
Is the development 'open source' ? Is there anyway we can help ?

Sent from my VTR-L09 using Tapatalk
The intention is to keep a closed source for the core, but opening the protocol as soon as it will became stable (in between RC and Release). Not because I want to protect anything (no rocket science used) but just to avoid the proliferation of forks typical of the opensouce.
Don't get me wrong on this: I fully support the opensource concept in many domains. I just think that the flight sim community is "too small" to take a real advantage from opensource. Modularity is the key.
I may be wrong, and other people are following different paths (thank you Ciribob for SRS). I will open the source the day I will not be able to support it anymore.

Going back to helping: any experience with iPad development? Android is not a problem, but Apple ecosystem is far from my area of expertise.
__________________

Author of DCSMP and VRK
Spoiler:
Ryzen 5 3600X - 32GB DD4 3200C14 Win10 64 - Geforce GTX 1080Ti
Hotas Warhog + Virpil T-50 base - Saitek Combat Rudder Pedals - Cougar MFCDs - Custom head tracker
35" UWQHD main display + 22" MFCD/Helios display / Rift S
2x256 GB SSD - 2Tb Caviar Green

Last edited by AMVI_Rider; 08-06-2019 at 09:58 PM.
AMVI_Rider is offline   Reply With Quote
Old 08-06-2019, 10:31 PM   #10
lucky-hendrix
Member
 
Join Date: Nov 2006
Posts: 344
Default

Makes sense, I was more asking to see if you need help. I would be afraid to see that kind of project slowly die because of lack of contribution


Unfortunately no, experience with apple ecosystem at all :/

Sent from my VTR-L09 using Tapatalk
lucky-hendrix is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT. The time now is 07:09 AM. vBulletin Skin by ForumMonkeys. Powered by vBulletin®.
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.