If I understand correctly, the flight originates from a non-tower airport within the 30NM Mode C veil, and terminates to an airport in Class D, but outside the Class B and associated Mode C veil.
The transponder requirement is not a VFR/IFR thing...if you have one installed, it must be checked every 24 months for compliance, and if not checked or found non-compliant, cannot be operated (14 CFR Part 91.413). This is a legacy requirement that originated with analog cavity tube transponder which exhibited a very predictable degradation in performance near end of life. Modern digital transponders do not show this sort of gradual decline towards minimum performance spec, but we are stuck with the requirement for the foreseeable future (blame TCAS and all those older cavity tube units that may remain in service until they can no longer be repaired... an unintended consequence of separate, ADS-B rule-compliant UAT-based ADS-B units).
If you are equipped with a separate ADS-B and transponder (such as a Mode A/C or S transponder and a separate UAT-based add-on unit), just placard the transponder as inoperative and comply with 91.215(d) by contacting the TRACON associated with your departure airport with your request... get the authorization from the TRACON and move the aircraft. I doubt there will be anything said, and you'll have the logged request and approval just in case you are ramp checked at destination.
If your ADS-B is integrated with the transponder, it may not be operated, so the sole procedure for requesting a deviation from the ADS-B Out requirement via the ADS-B Deviation Authorization Preflight Tool (ADAPT) is not permissible (use of the tool requires - amongst other things - an operable Mode A/C or S transponder with supporting 100' increment altitude encoder... yours is not operable). As mentioned above in another post, you'll need a waiver to the equipage requirement and that will take time and effort.
You may be able to remove your transponder and take it to your servicing avionics shop, or see if an inspected unit is available as a loaner. For an ADS-B capable transponder loaner, keep in mind that the aircraft personality module or internal aircraft-specific data (ICAO unique ID, tail, etc.) must be reprogrammed, and the flight check required for ADS-B out compliance may have to be redone (your avionics shop will have the current spin on policy available), but if the transponder side of the house is working, you meet the requirements to use ADAPT to obtain a deviation from the ADS-B requirement to exit the Mode C veil. You may also be able to find an avionics tech with an on-aircraft box tester and pay the bill associated with what looks like a house call. Ouch. Not an avionics tech - just an avionics engineer - check with your avionics shop and possibly the FSDO (if feeling particularly feisty) for more ideas. Also, 91.413 discusses integrated systems testing, so that may preclude a loaner depending on what the interpretation of that policy might be.
Obviously, the Class D is not an issue related to the transponder or ADS-B, assuming it resides outside of Class B/C and Mode C veil, etc.