I think the phrases "much faster" was used to compare the YF-23 to the YF-22 by Paul Tackabury, manager of YF-23 test & evaluation, and with it being potentially supercruising at Mach 1.8, which is 0.2 M faster than the YF-22's Mach 1.58, that is considerable. That said the difference between the F-23A and F-22A is much narrower, and Advanced Tactical Fighter to F-22 Raptor book by Aronstein, Hirschberg, and Piccirillo (one of the ATF SPO managers and was on the decision board) characterized the difference as "slight".
As far as weight goes, I think the F-23 not having to contend with thrust vectoring loads helped with weight, not just in the nozzles, but also the airframe structure to handle the vectoring loads. The F-15 STOL/MTD aircraft identified the need for reinforcements of the aft fuselage to handle the vectoring loads. Also, with most of the aircraft volume being in front of the wing's trailing edge, the horizontal tails of the F-22 extend well past the main fuselage structure on booms, which needed to be pretty strongly built in order to handle the loads as well as mitigating ballistic vulnerability.
Northrop also did seem to be much more aggressive in terms of composite usage on the F-23, but I have to wonder if this wounded up being a major risk area. The F-22 did not use as much composites as hoped for, something Sherm Mullin acknowledged in his interviews, and during EMD unforeseen issues, such as live-fire testing of the wing structure and tail fin buffeting, both of them resulting in increased usage of titanium in lieu of composites. Thermoplastics were largely supplanted by thermosets as well. This isn't to say that the F-23 would have ran into the same issue since aircraft development issues can't be transplanted like that, but I would wager that it's another area of high risk, similar to the risks of aft deck strength and heating. Given the budget pressure it would have faced in the 1990s, while it's not certain, I wouldn't be surprised if the F-23 design would have also experienced weight growth.