Idol of the Life-Binder

The stacks don't seem to work properly. Not sure if I'm misunderstanding or if its not working correctly. Currently have 4 Alex gems slotted, 3 on necklace and 1 on a ring. Still getting the same stacks as if I only had one gem. Mine reads gain 46crit per gem and with four gems I should be seeing 184crit per stack. I'm only getting 46crit per stack. I spend a ton of money getting gem slots because it looking like this item ramped up good. Not worth the money spent.

Edit: So it was a reading comprehension issue. When the idol procs, you earn a stack for each gem you have meaning you get the the secondary buff quicker. So for me to get to the 18 stacks needed it would have to proc 5 times (4gems x 5 procs = secondary buff). I'm still not sure if its worth.