Our symptoms were that both 4GB cards were recognised and the first cold boot would work but it would subsequently crash after a while and reboots would fail with the faulty memory chime. I have not retested it with 2 x 4GB since upgrading it to Mavericks. If its relevant originally the Mac mini was running 10.7.5 and has since been upgraded to 10.9.5. I ended up buying a single 8GB memory card with the intention of just using it alone in one slot but my colleague fitted it along side one of the 4GB cards and this has worked fine since as a total of 12GB. The Crucial pair also worked fine as a pair in the second Mac mini.
I also tried swapping with another pair which worked as a pair in another identical Mac mini but the problem then still happened in the first Mac mini.
Either one worked by itself in either slot just like you. The memory was a matched pair of 2 x 4GB bought from Crucial. I have one Mac mini 2012 which I had the same problem with. Crucial for Apple RAM 16GB (2x 8GB) DDR3 204-Pin SODIMM for Mac mini Core i7 2.3 GHz Late 2012 MD388LL/A (Crucial CT8G3S160BM Equivalent) CAS Latency: 11 Voltage: 1.