There could be two thing:
1: Probably your policy address has ip with /24 subnet. Please be specific with /32 IP in policy address, if you want to control single hosts.
e.g set address Trust "test" 10.1.1.0/24 <-- this is for /24 subnet
set address Trust "test/32" 10.1.1.1/32 <- this is for /32 or for single IP 10.1.1.1
refer KB https://kb.juniper.net/InfoCenter/index?page=content&id=KB15074&actp=search for more details
2: Make sure your specific policy are above of generic once.
e.g. : policy 1 which is for 192.168.0.0/24 <===>192.168.1.0/24 will trigger first than policy 2 which is for 192.168.0.1/32 <===>192.168.1.1/32 though it has more specific IPs. You can move policy 2 on top of policy 1 to work correctly.
Thanks,
Vikas