I think that's exactly how it works.
The math probably doesn't add up because each time you upgrade a point, it costs more to upgrade it another point. It's possible that the two points that were upgraded both cost 2082 and the next two points to upgrade cost 2162.