Jump to content

leapMotion implementation possibility


Recommended Posts

i tried the flyinside standalone flight simulator lastnight and its a step forward but i still think the leap motion is to limited in tracking ability to acuratly control a cockpit while flying. i could push the throttles and stuff like that alot better but then u go to grab your joystick and u hit about 5 switches on the way. i personally think you would be better off trying to implement leap motion hand models for the simple use of signals to wingmen in formations. good work though.

Link to post
Share on other sites
  • 1 month later...
  • Replies 265
  • Created
  • Last Reply

Top Posters In This Topic

Hi everybody,

 

as I said, I had a busy autumn at work.

I was able to get back to the project recently and today I have solved a major problem with the reset pilot's position function :drink:.

I was having this problem since the beginning of the project. Now that it's resolved, I'll be able to move faster.

 

Currently, I managed to:

- generate command exports to DCS;

- create simple buttons;

- create levers;

I now have to create the rotary knobs.

 

I hope to publish soon a video of my progress and why not create a deposit on Github so that those who are interested participate with me on this project.

 

Bye

Corei7 2600K 4.4Ghz OC - NVIDIA GTX 1080 Ti 11Gb - 16 Gb DDR3

HP Reverb PRO HMD - Thrusmaster TWCS + TopGun Stick

Link to post
Share on other sites
Hi everybody,

 

as I said, I had a busy autumn at work.

I was able to get back to the project recently and today I have solved a major problem with the reset pilot's position function :drink:.

I was having this problem since the beginning of the project. Now that it's resolved, I'll be able to move faster.

 

Currently, I managed to:

- generate command exports to DCS;

- create simple buttons;

- create levers;

I now have to create the rotary knobs.

 

I hope to publish soon a video of my progress and why not create a deposit on Github so that those who are interested participate with me on this project.

 

Bye

 

 

:thumbup:

 

 

:pilotfly:

Link to post
Share on other sites
  • 2 months later...
  • 4 months later...

Bumping back to the top, pretty keen to see if anyone can get something going with Leap Motion. All other options seem to be massive compromises for VR.

i7700k OC to 4.8GHz with Noctua NH-U14S (fan) with AORUS RTX2080ti 11GB Waterforce. 32GDDR, Warthog HOTAS and Saitek rudders. HP Reverb.

Link to post
Share on other sites
Bumping back to the top, pretty keen to see if anyone can get something going with Leap Motion. All other options seem to be massive compromises for VR.

 

Agreed!

 

But for me (cockpit builder) all I need is to be able to see my hands, I'll be operating real switches but I'm tired of having to feel around for them because my hands are invisible.

 

I'm chasing this down via multiple channels, now I'm about to check out the program this guy said he was using to do the overlay since an overlay is all I need. I'd like to be able to use Vive Trackers though, as well as Leap Motion.

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites

I'm chasing this down via multiple channels, now I'm about to check out the program this guy said he was using to do the overlay since an overlay is all I need. I'd like to be able to use Vive Trackers though, as well as Leap Motion.

 

Hi there,

just to add to the discussion, a possible solution would be to emulate valve's controller with the leap motion. This guy did it, and it's seems to work "nicely":

 

His implementation is available here.

 

According to the code, it should be possible to remove any controller inputs and just keep the finger tracking.

 

However, it still leaves you at the mercy of unwilling cockpit interaction because of hand/cockpit collision coded by ED. But if such problem were to be solved (I do hope it will) then this would remove the need to go through an overlay. Thus, you'll also have depth test on hands vs cockpit (such that a hand behind a cockpit feature will stay behind, which is not possible with an pure overlay).

Then, if ED ever attaches arms to the VR hands and add lighting interactions (shadows etc..), VR with a physical cockpit would truly shine :).

Link to post
Share on other sites

Well, perhaps I'd consider trying again but I fought with SDraw/Driver_Leap for hours the other day and it was a frustrating experience.

 

It was extremely unreliable as far as the virtual steam controller appearing, and to use it in DCS I had to hold my hands up in front of my face as soon as I click launch on the DCS.exe and keep them there until DCS loaded up or I'd have no controllers at all. That technique was recommended by the developer of the driver.

 

Then again, that video is only a month old so maybe worth another try.

 

Sometimes it had one controller active, sometimes six! And as you say, there's still the problem with DCS which hopefully they will solve but I don't want to wait without knowing if it might be years until they fix it. I will do a bug report on it and see if I get any response.

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites

That's good to know you already tried SDraw, there wasn't any mention of it in the thread.

 

It's a shame, I was hoping for it to be a good answer to hand tracking in VR.

 

Do you have any guess as to what is the root problem for these weird behavior?

Link to post
Share on other sites

I'd say it just needs to be debugged, and isn't getting enough dev time. I'd go for a payware version if it's easy to use and reliable.

 

Unfortunately Driver4VR Leap Motion support is also apparently a mess based on what I've read. I didn't buy it to try since lots of people were asking for refunds for that software.

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites

Thanks for helping bring it back to the top guys, still seems like there’s a way to go till someone like me can use it but maybe it’s not out of the question one day.

i7700k OC to 4.8GHz with Noctua NH-U14S (fan) with AORUS RTX2080ti 11GB Waterforce. 32GDDR, Warthog HOTAS and Saitek rudders. HP Reverb.

Link to post
Share on other sites

Well I guess LeapMotion is more interested in having their own implementation for hand/gesture interaction rather than having to fit into the OpenVR specification.

Ultimately, you can't blame them for that, mainly because the OpenVR specs are focused on interaction for physical devices (button pressed/released and so on) rather than modelling a virtual hand.

Link to post
Share on other sites
I use the BenOtterOVRlay http:// https://github.com/benotter/OVRLay.

Icebeat, you’re right, sharing this project is probably the best way to make it progress quickly and surely better. Indeed I’m very busy a work these past weeks and this fall I will be very busy.

Is there anyone comfortable with the github platform ??

 

Kariyann71 could you share your setup for the OVRLay?

 

I have no idea what I'm doing LOL, but I'm just trying to see my leap motion (and/or Vive Tracker) hands in DCS. I don't want my hands to operate anything in DCS, I just want to see them in 3D so I can find switches more easily in a real cockpit.

 

Any help would be very much appreciated, and I know there are others wanting the same thing.

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites

https://media.discordapp.net/attachments/303984261264965641/598173821069033512/JPEG_20190709_172855.jpg?width=658&height=878

 

 

Just worked on this, it still needs some minor polish, but it's nearly finished.

i9 10920X @4.8GHz, X299X HEDT mainboard, RTX3090 OC, quad channel 64 GB RAM @3600, Valve Index, HOTAS & Rudder: all Virpil with FFB base mod (G940), DCS: all modules

Link to post
Share on other sites

Awesome! I hope you can provide installation instructions "for dummies"!

90% of the time of I find something I need is on github I can't figure out how to use it.

 

I'd happily pay $20 usd for this of its easy to install and will be maintained so it keeps working when steam VR does updates.

 

Ideally (wish list here, not mandatory) with it would also have adjustable scale as well on case your home cockpit is a little later or smaller than in game.

 

You can adjust the cockpit size in DCS with force IPD but it would be better to scale the overlay up or down a bit.


Edited by TripRodriguez

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites

This looks amazing! Hope to see the finished project. Getting my Index Friday.

 

https://media.discordapp.net/attachments/303984261264965641/598173821069033512/JPEG_20190709_172855.jpg?width=658&height=878

 

 

Just worked on this, it still needs some minor polish, but it's nearly finished.

2080Ti FTW3 Ultra - G.Skill RJ 32GB (16x2) DDR4 3200 - Ryzen 2700X 4.2Ghz OC - Corsair H100i Pro - Samsung 970 EVO M.2 2TB - TMW HOTAS w Delta Sim - F/A-18C grip - 10cm Sahaj - TrackIR 5 Pro - Rift CV1 - MFG CWind - BuddyFox UFC - DSD RK II - Cougar MFDs w/ LCDs - Foxx Mounts - VPC MongoosT-50CM base

 

- Maps: NTTR, Persian Gulf, Normandy

- Modules: FC3, F-14A/B, F/A-18C, AV-8B, A-10C, F-16C, F-86, KA-50, P-51D, WWII assets, and [insert campaign name]

Dreaming of the F-15E / F-14D / Rhino

Link to post
Share on other sites
This looks amazing! Hope to see the finished project. Getting my Index Friday.

 

I've got mine mounted too. I got the cord coiled in the frunk

 

Alec Delorean were you just talking about mounting your Leap, or did you mean you are working on the software? I was on mobile at the time with almost no signal so I didn't see until now that it was just a pic of the Leap on the Index.

 

Software is the problem.....


Edited by TripRodriguez

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites

Hi everybody,

 

I stopped this program cause I finally found it was not appropriate to DCS, especially considering World scaling.

I think a « mix » leap motion and hardware button would be more appropriate.

Nevertheless I could put my project in this thread if someone want to continue this one.

Corei7 2600K 4.4Ghz OC - NVIDIA GTX 1080 Ti 11Gb - 16 Gb DDR3

HP Reverb PRO HMD - Thrusmaster TWCS + TopGun Stick

Link to post
Share on other sites
Hi everybody,

 

I stopped this program cause I finally found it was not appropriate to DCS, especially considering World scaling.

I think a « mix » leap motion and hardware button would be more appropriate.

Nevertheless I could put my project in this thread if someone want to continue this one.

 

Please do, "« mix » leap motion and hardware button" is exactly what I'm wanting to do but I don't know how to use Unity, most of the time I find something I need on Github I don't know how to use it. All I can do is hope for detailed instructions or someone who knows more and wants to help.

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites
  • 2 weeks later...
Please do, "« mix » leap motion and hardware button" is exactly what I'm wanting to do but I don't know how to use Unity, most of the time I find something I need on Github I don't know how to use it. All I can do is hope for detailed instructions or someone who knows more and wants to help.

 

Seconded. I have the leap motion and want to use it to help steer my hands towards hardware buttons.

Link to post
Share on other sites

Completed a proof of concept test tonight for my best sim cockpit solution so far which is using Vive Trackers via OVRIE. Here is a (REALLY CRAPPY and therefor unlisted!) video. Don't mind the mess around the sim, parts, sawdust, and tools everywhere.... For this video to really be worth anything I'd need to have captured the gameplay to put up side by side. I'll try to get that done later in the week. For the first bit of the video you can (just barely) see the monitor in the background so you can see my DCS hand.

 

Explanation: For several years on and off I've been trying to find a satisfactory solution to using VR in a full simulator cockpit. I find that my immersion is completely ruined by having to fumble blindly to find the buttons and switches that are right in front of me because I can't see my hands.

 

In this video, I have the beginning of my Huey sim cockpit, an (unfinished) complete forward console. I'm wearing the VR headset thorughout the video. I have used OVRIE to have a tracker attached to my hand emulate an Index controller, and I've used the OVRIE offsets functionality to get my hands orientation and position matched up pretty precisely.

 

The hand is not animated at all, but the 3D model is in a pretty standard relaxed hand pose so it's easy to just relax my hand so it matches. I have aligned the real switches and knobs so that when I put the middle knuckle of my VIRTUAL index finger on the right side of the switch or knob the same joint of the same REAL finger is in the same place.

 

In the video I demonstrate how quickly and accurately I can find my knobs and switches. It's very satisfactory. Now the only problem is I have to worry about OVRIE getting broken. =(

 

[sIGPIC][/sIGPIC]

Demo of my 6DOF Motion VR Sim:

Link to post
Share on other sites
  • 1 month later...

Based on my previous work, I created a new solution.

This is not a full VR solution as the first one, this is more a VR mouse with some handsignals commands.

Currently mouse is working nicely, I’m continuing some tweaking but I expect a release in the next weeks. Clicks based on pinch and index movements.

About handsignals I succeeded in command catapult launch with salute gesture, I’m working on ejection command,power supply and catapult alignement.

 

Hope coming back soon

Corei7 2600K 4.4Ghz OC - NVIDIA GTX 1080 Ti 11Gb - 16 Gb DDR3

HP Reverb PRO HMD - Thrusmaster TWCS + TopGun Stick

Link to post
Share on other sites
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...