Anomaly:
Thx read up on it a bit more.
So as soon as you work a 6 day week (hgv and including other work) you need a 45 hour continuous break before driving again. So I could work one day each weekend if I managed to get a Sunday job starting after 3pm 
Not quite. Each Driving Hours Week (where the start of a “Driving Hours Week” is defined as the point when you start work at the end of a weekly rest), you must start a weekly rest before the same time six days later. For example, if you take Sunday off, and start your office job at 9am on Monday, you must start a weekly rest before 9am Sunday. Now, a weekly rest period can be either a full one (45 hours), or a reduced one (at least 24 hours).
Also, in any two consecutive Calendar Weeks (Monday-Sunday), you must have at least one full (45-hour) weekly rest period.
So, the 6-day thing isn’t really relevant. It’s the fact that, by working on the sixth day, you’re reducing your weekly rest down to below 45 hours. For example, if you work Monday-Friday in the office, then do a driving shift on Saturday, you’ve probably had to reduce your weekly rest (from the end of your driving shift on Saturday until the start of the office job on Monday) to below 45 hours. That means that you’ll need to have a full 45-hour weekly rest the following week (unless you happen to squeeze it in some other time, like taking holiday, for example).
It also means that you’re unlikely to be able to work Sunday on one weekend and then Saturday on the following weekend, even if you’re able to squeeze in a 45-hour rest at the weekend, because your Saturday shift will have to finish before the time at which your previous Sunday shift started.
To add to what ROG said, you still need to have your daily rest after that shift. If your Sunday shift starts at 1500, then it’s only 9 hours to midnight, at which point you have to start your daily rest, or you will have to start work late on Monday morning (since you must have a minimum 9 hours reduced daily rest). Most driving jobs will require shifts over 9 hours, at least occasionally. In my experience, agencies will pay (and charge their clients) for a minimum 8-hour shift, so the client will generally expect you to work at least 8 hours. Given that, it’s easy to see why shifts are likely to go over 9 hours in many cases. Limiting yourself to a 9-hour shift (or even a 10-hour one starting at 2pm on Sunday, which is the most you can do legally), is likely to prove a significant limitation on the work you’re offered.
In my experience, you’re likely to end up being limited to working every other weekend in the vast majority of cases. I would aim for the early Saturday morning starts (you can legally start from 2am, but there tend to be a lot of jobs starting at 4am for some reason), because then, if the shift is short and finishes before midday, you can get a full 45-hour break in and work the following weekend, but if the shift runs long, you take a reduced weekly rest and then take the whole of the following weekend off. as a normal weekly rest. That extended weekly rest would also probably pay back any compensation for the reduced rest, in case you weren’t able to pay it back during the week (because it was too many hours in a block). You really don’t want to end up getting stuck somewhere having run out of hours, and wait until Monday morning before you can drive again 
If your office hours are flexible, then that gives you a bit more scope to move things around to squeeze the rests in (e.g. finishing early on Friday or starting late on Monday).