I'd have to agree with Jeff to choose XD over Glock for all calibers:
XDs have a better grip than the blocky finger spreading grooved Glock
XDs have a fully supported chamber to avoid the kB! failures of Glocks (the reason why Glocks explode)
XDs have internals made from billet
If interchangability is such a concern, consider a 5" XD compact with supplied grip extension to transform into XD tactical
FWIW, poly guns excel in 2 areas: weight and cost savings. Weight is critical for something you will be carrying all day long such as a CCW. Cost savings is up to what you can afford. But for home defense and range work, weight is less important. In fact, a heavier frame helps reduce felt recoil and muzzle flip. You might want to give these a try:
Beretta 92FS (15rd mag)
Para Ordnance P14-45 (14rd mag) - I'd take the Series 80 mush over the Glock spongy mush trigger.