360 degrees panoramic mouse pointer

As anyone using trackir or similar device to move the view knows, using the mouse to manipulate de cockpit switches, etc can become a torture and the tracking must be paused all the time.

 

Two days ago I realized that the problem comes from the fact that the mouse pointer uses screen coordinates instead of in-game coordinates and a mouse pointer that stays in place relative to the cockpit objects would be a lot easier to use because the movement of the head would not interfere.

 

And it really is that easier! I use my own software for head tracking with a GlovePIE script, and I decided to expand it to give yaw and pitch coordinates to the mouse pointer and then map it to screen coordinates. This way the cursor can be positioned at any place over the cockpit and stays there no matter the head movement.

 

I'll post the relevant lines of my script here so if anyone else is using custom tracking software can see how it's done and imitate it. And it's worth the try!!!! It's so easy to manipulate the cockpit with a pointer that stays meekly where you left it.

 

 

GlovePIE code:

var.mVelyaw=0.05    //Cursor speed for yaw position
var.mVelpitch=0.05         //Cursor speed for pitch position
var.fovx=50                 //In-game horizontal field of view
var.resx=1680              //Horizontal screen resolution
var.resy=1050               //Vertical screen resolucion
var.fovy=var.resy*var.fovx/var.resx  //In-game vertical field of view

if (starting)
  var.myaw=0   //mouse yaw coordinate (longitude)
  var.mpitch=0  //mouse pitch coordinate (latitude)
  //Calculate the "scale" of the screen
  var.c=cos(-var.fovx/2)
  var.rangox=|(sin(-var.fovx/2)/var.c)|
  var.c=cos(var.fovy/2)
  var.rangoy=|(-sin(var.fovy/2)*cos(-var.fovx/2)/var.c)|
  mouse.SwallowMovement=true  //disable system mouse cursor update
endif

var.myaw+=(Delta(mouse.DirectInputX)*var.mVelyaw) deg  //Update mouse yaw coordinate from directinput mouse movement information
if (var.myaw>180)
  var.myaw-=360
else if (var.myaw<-180)
  var.myaw+=360
endif
var.mpitch+=(Delta(mouse.DirectInputY)*var.mVelpitch)  deg//Update mouse pitch coordinate
if (var.mpitch>180)
  var.mpitch-=360
else if (var.mpitch<-180)
  var.mpitch+=360
endif
var.c=sin(var.realpitch)*sin(var.mpitch)+cos(var.realpitch)*cos(var.mpitch)*cos(var.myaw-var.realyaw)
var.x=cos(var.mpitch)*sin(var.myaw-var.realyaw)/var.c
var.y=(cos(var.realpitch)*sin(var.mpitch)-sin(var.realpitch)*cos(var.mpitch)*cos(var.myaw-var.realyaw))/var.c
var.posx=EnsureMapRange(var.x,-var.rangox,var.rangox,-var.resx/2,var.resx/2)
var.posy=EnsureMapRange(var.y,-var.rangoy,var.rangoy,-var.resy/2,var.resy/2)

if (InRange(var.posx+var.resx/2,0,var.resx))
  mouse.CursorPosX=var.posx+var.resx/2
endif
if (InRange(var.posy+var.resy/2,0,var.resy))
  mouse.CursorPosY=var.posy+var.resy/2
endif

Constraints:

-It needs to know the field of view used, so if the information is not available on the fly it must be hard coded and that fov used in the simulator

-Change in position of the eye point (leaning left/right, etc) is not taken into account and the mapping from angular to screen coordinates fails when leaning the head in some direction.

 

_

 

So, my advice to developers is that if the cockpit can be manipulate with a pointer, make the pointer coordinates relative to the cockpit! It makes things a lot easier.

Edited February 19, 2011 by average_pilot

Sounds good.

Any idea how to make it with TrackIR?

In theory glovepie can read trackir devices. But I dont have a trackir to test.

I can direct you to do a test if you wish. If glovepie can read the data from your trackir the data can be use to manipulate the mouse pointer to create the effect.

var.mVelyaw=0.05    //Cursor speed for yaw position
var.mVelpitch=0.05         //Cursor speed for pitch position
var.fovx=50                 //In-game horizontal field of view
var.resx=1680              //Horizontal screen resolution
var.resy=1050               //Vertical screen resolucion
var.fovy=var.resy*var.fovx/var.resx  //In-game vertical field of view

if (starting)
  var.myaw=0   //mouse yaw coordinate (longitude)
  var.mpitch=0  //mouse pitch coordinate (latitude)
  //Calculate the "scale" of the screen
  var.c=cos(-var.fovx/2)
  var.rangox=|(sin(-var.fovx/2)/var.c)|
  var.c=cos(var.fovy/2)
  var.rangoy=|(-sin(var.fovy/2)*cos(-var.fovx/2)/var.c)|
  mouse.SwallowMovement=true  //disable system mouse cursor update
endif

var.myaw+=(Delta(mouse.DirectInputX)*var.mVelyaw) deg  //Update mouse yaw coordinate from directinput mouse movement information
if (var.myaw>180)
  var.myaw-=360
else if (var.myaw<-180)
  var.myaw+=360
endif
var.mpitch+=(Delta(mouse.DirectInputY)*var.mVelpitch) deg //Update mouse pitch coordinate
if (var.mpitch>180)
  var.mpitch-=360
else if (var.mpitch<-180)
  var.mpitch+=360
endif
var.c=sin(-trackir.pitch)*sin(var.mpitch)+cos(-trackir.pitch)*cos(var.mpitch)*cos(var.myaw-trackir.yaw*2)
var.x=cos(var.mpitch)*sin(var.myaw-trackir.yaw*2)/var.c
var.y=(cos(-trackir.pitch)*sin(var.mpitch)-sin(-trackir.pitch)*cos(var.mpitch)*cos(var.myaw-trackir.yaw*2))/var.c
var.posx=EnsureMapRange(var.x,-var.rangox,var.rangox,-var.resx/2,var.resx/2)
var.posy=EnsureMapRange(var.y,-var.rangoy,var.rangoy,-var.resy/2,var.resy/2)

if (InRange(var.posx+var.resx/2,0,var.resx))
  mouse.CursorPosX=var.posx+var.resx/2
endif
if (InRange(var.posy+var.resy/2,0,var.resy))
  mouse.CursorPosY=var.posy+var.resy/2
endif

if ((Released(mouse.LeftButton))&&(mouse.RightButton))
  mouse.SwallowMovement=!mouse.SwallowMovement
endif

This one is supposed to work with a trackir. It's the same script changing var.realyaw and ver.realpitch with trackir.yaw and trackir.pitch, that should read yaw and pitch from a trackir device. Someone with it should try.

 

I have added a condition to disable the "mouse.SwallowMovement" by clicking the left mouse button whilst the right button is held. In case the script is not working, or the swallowmovement command itself, the mouse could not be moved while the script is running, hence the need to be able to disable it.

 

Resolution and horizontal FOV must be modified according with each one settings. The cursor speed should be modified to each one likes.

The vertical fov (var.fovy) is calculated from the display resolution and the horizontal fov, but didn't fit well in the few tests I've done. That's why I put 25 degrees by hand for fovy in the first script I posted. I still have to see why it happens.

Edited February 23, 2011 by average_pilot

How do you get this to work with TIR? This is one of the biggest issues with the clickable pit and searching there's a lot of threads on this.

I'm not using TIR, but custom software and a wiimote IR cam, so I can feed the glovepie script with the yaw and pitch angles of the in-game camera.

Glovepie can read data from a TIR device, so I had hopes that the second script I posted could work with TIR users. I can't test it by myself as I don't own any version of TIR.

 

Right now I'm about to edit those scripts. I found out that the way I was mapping the mouse position to the screen was more like a cilinder, not a sphere, and mouse cursor offsets a lot when looking down or up. After some research to know how to properly project a sphere (panoramic mouse "space") into a plane (screen), I found the equations for what it's called an gnomonic azimuthal projection, one of the projections methods to do maps of the Earth.

 

Still not perfect, because it seems that the axis of rotation of the eye point is not at that eye point position, so the eye point moves left/right and up/down when looking left/right/up/down, even disabling X/Y/Z motion of the head-tracker. The projection is always done at the center of the sphere. I have no idea how to do that projection of a sphere from an arbitrary center of projection, nor can't find it at internet. Any mathematician would do it easily, as it doesn't seem like a complex problem to resolve with the aproppiate knowledge.

 

An additional problem I'm finding is that the vertical field of view of the game is not what I should expect from the aspect ratio of the resolution I use. It's like if the pixel aspect ratio is not 1:1, or more probably that I haven't edited properly the config file that unlocks the angles of the cockpit view.

 

I'm using it in FSX too and it works like a charm.

Despite all, in Black Shark also makes things a lot easier.

 

Still, the best would be to have it managed inside the game engine itself.

Doesn't have to be perfect, just iron out the small jitters as you look at the control you want to press.

I think this idea does have some merit, but the problems you've already run in to can't be completely avoided. The "pilot's eyes" are free to move around within the cockpit. Your "spheric projection" of the mouse pointer will either have to remain stationary, or follow the pilot's POV. In either case, the 3 important points (POV, mouse pointer, and expected activation point (whichever cockpit control you're trying to press) will not remain in a line as the POV moves - which seems to be your goal.

 

In order to make this work as intended, you'd have to "project" the mouse pointer onto a projection of the cockpit itself (as close as possible to the model used in-game).

 

I have no doubt that your method does reduce the jitter significantly - and suspect that there's a very good balance that could be achieved - perhaps adding a function to adjust the scale of the stationary (relative to either the cockpit or POV) sphere (where you calculate the relative screen coordinates of the mouse pointer).

 

I may not have explained that very well ...

You have eplained it very well, indeed. You've exposed the problems I tried to explain before but in a clearer language.

 

Now the projection is calculated with the center of the sphere as the center of projection. So, next step would be to find the equations to project it from any arbitrary point. That would be enough for most situations.

And the ultimate solution would be to have a rough 3D model of the cockpit and project this model onto the screen instead of a sphere. But I don't feel like doing so much work. I'm more interested in the second solution.

 

As I said, it works very well in FSX when I disable the X/Y/Z movements, because in FSX I can know with precision the orientation of the camera and the field of view. In Black Shark I'm using 5 axis of a virtual joystick (ppjoy) assigned to yaw/pitch/X/Y/Z, so I must deduce the orientation from the range of movement (360 deg for and 180 deg for pitch assuming I have done the tweaking correctly in the correct lua file) and the position of the corresponding axis, but I think I'm not getting accurate results like in FSX.

 

What would be great is if someone knowledgable in geomery would post the equations to project a sphere from an arbitrary point. Then I could change the current ones with those ones and see what happens.

I'm certainly no geometry whiz ...

 

But what about a slightly different approach? Instead of trying to project the mouse pointer onto a sphere, why not simply apply a negative offset relative to head-tracker input? Turn your head to the left, mouse pointer "moves" to the right, keeping it 'in place' relative to the cockpit. Input from the mouse would be handled normally.

That's very much what I first did. The problem was that the speed at which everything moves when you pan your view left or right is greater when looking at the horizon level than when looking at your knees. That's why I searched for another solution and changed the scripts.

Yesterday I did another flight in FSX and it really works well. I'm liking it so much. Unfortunately I'm not having time to tweak Black Shark calmly and achieve the same level of accuracy. I've read that I can use the latest dll from trackir software to make the glovepie's trackir emulation work with Black Shark. That would make things a lot easier for me.

 

I think I will do a little video showing how it feels to have a mouse cursor that is static with respect to the cockpit. Maybe this way someone with a trackir would try the script to see if it works.

I'd try it, but right now I only have beta4 and it's unstable for me, it's been a hectic week and thinking I'll wait for beta6 to appear before I do a clean install

I have a TrackIR, but I'm not familiar with what I'd need to do to try the script.

You need to download glovepie from glovepie.org. It doesn't need any installation. Just unzip on a folder of your liking and launch the exe.

 

Copy/paste the script on the editor window and click the "Run" button. If the mouse move by itself when you move your head while trackir is enabled, then it works. Otherwise, it doesn't, and the universal solution is not possible.

It moves the cursor vertically, but it jumps wildly when moving side to side.

That doesn't sound bad, because it means that it's receiving data from the trackir.

I've added a variable to toggle the effect, and now it's off by default to be able to use the mouse as usual before entering the simulation. Click the mouse left button while holding the right button to toggle it.

 

If you'd wish to do so, try it. Change var.resx, var.resy and var.fovx to the values you are using.

var.mVelyaw=0.05    //Cursor speed for yaw position
var.mVelpitch=0.05         //Cursor speed for pitch position
var.fovx=50                 //In-game horizontal field of view
var.resx=1680              //Horizontal screen resolution
var.resy=1050               //Vertical screen resolucion
var.fovy=var.resy*var.fovx/var.resx  //In-game vertical field of view

if (starting)
  var.myaw=0   //mouse yaw coordinate (longitude)
  var.mpitch=0  //mouse pitch coordinate (latitude)
  //Calculate the "scale" of the screen
  var.c=cos(-var.fovx/2)
  var.rangox=|(sin(-var.fovx/2)/var.c)|
  var.c=cos(var.fovy/2)
  var.rangoy=|(-sin(var.fovy/2)*cos(-var.fovx/2)/var.c)|
endif

if (var.enabled)
  var.myaw+=(Delta(mouse.DirectInputX)*var.mVelyaw) deg  //Update mouse yaw coordinate from directinput mouse movement information
  if (var.myaw>180)
     var.myaw-=360
  else if (var.myaw<-180)
       var.myaw+=360
  endif
  var.mpitch+=(Delta(mouse.DirectInputY)*var.mVelpitch) deg  //Update mouse pitch coordinate
  if (var.mpitch>180)
     var.mpitch-=360
  else if (var.mpitch<-180)
       var.mpitch+=360
  endif
  var.c=sin(-trackir.pitch)*sin(var.mpitch)+cos(-trackir.pitch)*cos(var.mpitch)*cos(var.myaw-trackir.yaw*2)
  var.x=cos(var.mpitch)*sin(var.myaw-trackir.yaw*2)/var.c
  var.y=(cos(-trackir.pitch)*sin(var.mpitch)-sin(-trackir.pitch)*cos(var.mpitch)*cos(var.myaw-trackir.yaw*2))/var.c
  var.posx=EnsureMapRange(var.x,-var.rangox,var.rangox,-var.resx/2,var.resx/2)
  var.posy=EnsureMapRange(var.y,-var.rangoy,var.rangoy,-var.resy/2,var.resy/2)

  if (InRange(var.posx+var.resx/2,0,var.resx))
     mouse.CursorPosX=var.posx+var.resx/2
  endif
  if (InRange(var.posy+var.resy/2,0,var.resy))
     mouse.CursorPosY=var.posy+var.resy/2
  endif
endif

if ((Released(mouse.LeftButton))&&(mouse.RightButton))
  Toggle(mouse.SwallowMovement)
  Toggle(var.enabled)
  var.mpitch=0
  var.myaw=0
endif

Edited February 23, 2011 by average_pilot

Same ...

 

Vertical movement is precise, decently matched to head movement.

 

Horizontal movement is unusable. As soon as my head turns off-center (left or right), the mouse cursor goes crazy.

This shows potential.

Patiently awaiting for some solution for TrackIR.

Ingenious... ED, take note! A mouse cursor that positions itself relative to the 3d world, and not the screen. What a concept! Rep.

A pity.

I've found a little typo in the script, but it didn't affect the overall result.

 

I'd need to debug it in my own computer to see variables and the like. But I'm not going to buy a trackir in the near future, it's an extremely overpriced piece of hardware. I have almost the same with the cost of a wiimote and 3 IR LED's.

: /

 

There is a little chance. If I find the way to make Black Shark work with trackir emulation I could see why it behaves so different. My guess is that it doesn't use the angle from trackir as absolute direction. That would mean that tweaking Black Shark to change the angle locks of the cockpit view affects the script, as it expects that 10 degrees to the right from trackir means 10 degrees to the right of the cockpit camera and so on.

But it's just a guess.

Book example of thinking outside the box :thumbup:

wow... I thought that the problem with trackir was just a compatibility question, but it's because the interface with trackir is now encrypted. It looks like everybody is playing dirty lately and is adding all kind of artificial incompatibilities via software.

 

That typo I mentioned in the last post turned out to be one of the problems. I judged it bad.

 

Ok, this is copy/paste of what I'm using right now and it's working quite well. This will probably be my last try of giving to the masses what I'm enjoying now in my computer.

If someone cares to try, remember that it's important to type the resolution and the exact field of view in the script in the variables resx, resy and fovx.

 

if (starting)
  var.mVelyaw=0.025
  var.mVelpitch=0.025
  var.fovx=50.4
  var.resx=1680
  var.resy=1050
  var.fovy=var.resy*var.fovx/var.resx
  var.myaw=0
  var.mpitch=0

  var.c=cos(-var.fovx/2)
  var.rangox=|(sin(-var.fovx/2)/var.c)|

  var.c=cos(var.fovy/2)
  var.rangoy=|(-sin(var.fovy/2)*cos(-var.fovx/2)/var.c)|
endif
if ((Released(mouse.LeftButton))&&(mouse.RightButton))
     Say("Panoramic mouse toggle")
     mouse.SwallowMovement=!mouse.SwallowMovement
     Toggle(var.panmenabled)
     var.myaw=0
     var.mpitch=0
endif
var.myaw+=(Delta(mouse.DirectInputX)*var.mVelyaw) deg
if (var.myaw>180)
  var.myaw-=360
else if (var.myaw<-180)
    var.myaw+=360
endif
var.mpitch+=(Delta(mouse.DirectInputY)*var.mVelpitch) deg
if (var.mpitch>180)
  var.mpitch-=360
else if (var.mpitch<-180)
    var.mpitch+=360
endif
var.realyaw=trackir.yaw*2
var.realpitch=-trackir.pitch
var.c=sin(var.realpitch)*sin(var.mpitch)+cos(var.realpitch)*cos(var.mpitch)*cos(var.myaw-var.realyaw)
var.x=cos(var.mpitch)*sin(var.myaw-var.realyaw)/var.c
var.y=(cos(var.realpitch)*sin(var.mpitch)-sin(var.realpitch)*cos(var.mpitch)*cos(var.myaw-var.realyaw))/var.c
var.posx=EnsureMapRange(var.x,-var.rangox,var.rangox,-var.resx/2,var.resx/2)
var.posy=EnsureMapRange(var.y,-var.rangoy,var.rangoy,-var.resy/2,var.resy/2)
if (var.panmenabled)
  if (InRange(var.posx+var.resx/2,0,var.resx))
     mouse.CursorPosX=var.posx+var.resx/2
  endif
  if (InRange(var.posy+var.resy/2,0,var.resy))
     mouse.CursorPosY=var.posy+var.resy/2
  endif
endif

Did anyone ever test this with Freetrack or FacetrackNoIR?

 

Where do we get the Horizontal FOV from to put in as fovx? Is it in a specific DCS config file?

Where do we get the Horizontal FOV from to put in as fovx? Is it in a specific DCS config file?

I think it's in server.lua and the variable is ViewAngleLimits. If you set the same minimum and maximum limit the FOV will be fixed. But, in my experience, the number you put in server.lua is not exactly the fov you'll have in the game by several degrees. When I had the time and patience for all these things I used to edit server.lua and then measure the in-game fov with another script and go back to server.lua.

For example, for 50º I had to put something like 54 in server.lua (just a example, I don't remember any exact numbers). I don't know the reason for that.

 

One more thing, the formula to calculate the vertical fov (var.fovy) from the horizontal is incorrect!!!! I thought at that time that the same proportion between vertical and horizontal resolution applies to the vertical and horizontal fov, but it's not true.

I think it's in server.lua and the variable is ViewAngleLimits. If you set the same minimum and maximum limit the FOV will be fixed. But, in my experience, the number you put in server.lua is not exactly the fov you'll have in the game by several degrees. When I had the time and patience for all these things I used to edit server.lua and then measure the in-game fov with another script and go back to server.lua.

For example, for 50º I had to put something like 54 in server.lua (just a example, I don't remember any exact numbers). I don't know the reason for that.

 

One more thing, the formula to calculate the vertical fov (var.fovy) from the horizontal is incorrect!!!! I thought at that time that the same proportion between vertical and horizontal resolution applies to the vertical and horizontal fov, but it's not true.

 

OK thanks, all sounds a bit complicated for me ;)

 

I'm guessing that if we set the min and max fov the same it prevents us moving/zooming forward at all doesn't it, which would be a problem, so maybe I need to wait and hope that ED can implement something like your idea without this restriction?