Maybe something like this would help.
http://www.blitzbasic.com/codearcs/codearcs.php?code=1870
Not tried that routine myself but any 3d point distance to line routine with appropriate parameterisation should do. (there is more than one way to define a line, and you want to use a routine that expects a start xyz and an end xyz). Given the tformed picked coords of a point and a picked triangle select the pair of verts (0>1,1>2,2>0) based on the distance of the point from the edges they define.
http://www.blitzbasic.com/codearcs/codearcs.php?code=1870
Not tried that routine myself but any 3d point distance to line routine with appropriate parameterisation should do. (there is more than one way to define a line, and you want to use a routine that expects a start xyz and an end xyz). Given the tformed picked coords of a point and a picked triangle select the pair of verts (0>1,1>2,2>0) based on the distance of the point from the edges they define.