I'm looking for info about just how much lift is lost to a wing tip (i.e. 2D model vs. 3D reality), and how much a well designed wing-tip fence can mitigate that effect. Any thoughts, links, references, etc. are appreciated.
Easy question, slightly involved answer... As usual, the answer is "it depends!"
Some background terminology, in case you haven't seen this elsewhere: Engineers usually describe the amount of lift a wing is producing in terms of a unitless "lift coefficient." The 2D airfoil results are computed as C_l = l / (q*c), where l (lowercase L) is the lift force being produced per unit span, q is the "dynamic pressure" = 0.5*air density rho * farfield airspeed V squared, and c is the wing chord. Underscores _ indicate subscripts. Similarly, 3D wing results are presented as C_L = L / (q*S), where L is the total lift force produced on the wing, q is as before, and S is the wing reference area.
So, I think what you're asking is how does 2D C_l relate to 3D C_L ? That ratio C_l / C_L varies with another parameter, the wing "aspect ratio" AR= b^2/S, where b is the wing span and S the area again. As aspect ratio approaches infinity, the ratio C_l / C_L approaches 1. For a "high" aspect ratio (think sailplane, >20), the ratio might be around 0.9. For a "low" aspect ratio like 3, it will be much less. I haven't found an equation I trust enough to link to online...
Regarding wing tip devices, fences/winglets don't do much at all for lift, but can help reduce the "induced" drag produced by generating lift if carefully designed. See Ilan Kroo's online textbook here for some details on winglets. Nonplanar Wings and Winglets (the rest of section 12 may also be relevant to your questions)
The equation I've seen on the net a lot is 3D CL = 2D Cl * AR / (AR + 2). One site even had a derivation of it. I don't know how accurate it is, though I have heard there are other variables at play.
I don't know how much winglets would affect CL. I would assume you would add their height to the span in your calculation of AR. Even if they do better than the span extension, that equation sure is not generous about letting you get near 100% at any AR.
The presence of one curved surface near another curved surface could cause boundary layer interference, which could affect lift, so I doubt the lift calculation for a winglet can be calculated with a simple formula. It probably is done on a case by case basis.