Although commonly stated an engine needs some backpressure, that's not really the case.  The loss of backpressure will create driveability problems, but not due to the backpressure it's self.  When the exhaust is opened up, backpressure drops, and low end drops, but here is why.

An object in motion stays in motion.  When exhaust is rushing out of the exhaust valve and through the exhaust system, there is a given velocity to the exhaust gasses.  The larger the pipe the more area, so the less volecity.  The smaller the pipe the less area, and the greater the volecity.  It can be compared to the narrow part of a river flowing faster, while the wide part just moseys along gently.

The object in motion, when moving faster or at greater velocity, creates a greater force.   When the exhaust valve shuts, the gasses try to keep moving.  This creates a vacuum pulse at the valve and in the exhaust system.  The rapidly moving gasses try to keep moving but have no source, so they create a vacuum while trying to get further away from the engine.

This vacuum pulse will be much greater with small exhaust opening, and much less with larger openings.  The vacuum pulse is an advantage.  With vacuum present, when the next valve opens the vacuum actually helps to "suck" the exhaust out of the cylinder.  The greater the suction, the more old exhaust gasses are removed.  A bit of valve overlap from the cam helps this work.   And the more old stuff you can get out, the more new stuff you can get into the cylinder.  That means the cylinder can create more power.

Go to a larger exhaust, and you lose the velocity and vacuum pulse, and thus lose the engine scavenging, and lose low end torque.

At highter engine rpm, the velocity of the exhaust is increased, and the engine can now again perform better.

The flip side is at full open throttle and high rpm, the smaller exhaust can be too restrictive to flow, and create the cylinder purging problem at high rpm.  You just can't get enough out of the cylinder fast enough, and high rpm suffers.

The camshaft will also create a great deal of control.  A very agressive cam has more duration and overlap to allow full purging of the cylinder at high rpm.  But this hinder the same thing a slow rpm.  Poor purging causes the "Lope" of high performance cams.  They just don't purge the cylinder properly at slow speeds.

Combining the cam and exhaust systems, along with the intake, creates the best for either low end or high end, or mid range.  There is always some trade off.

The correct size headers help even more by cutting restrictions at high speed while equally distributing the vacuum pulse to each cylinder to further enhance cylinder breathing.  The length of the header tube is just as important as the diameter to control flow.

The key is vacuum and flow rates, not backpressure.  Backpressure will keep gasses trapped in the cylinder, and is counterproductive to purging the cylinders.  An open leaking exhaust will have no flow/vaccum operation.