Right, this will work that way. Think that you only have two ibgp peers, will reflect each other, so will not see any different behaviour to configure R4 in the same cluster.
R1 will reflect R3 prefixes to R4, and R4 prefixes to R3. But R3 is a RR client and R4 not. This would be a strange behaviour. The answer, R4 is outside the RR cluster. So prefix from inside cluster are reflected outside cluster, and from outside to inside.
If you add a R5, R6 routers, outside cluster, will reflect R3 routes to every ibgp peer outside cluster. But will not reflect R4, R5, R6 prefixes between them, because they are not in a RR cluster.
Look, the following changes to my configuration. Will reflect routes inside cluster and outside cluster.
delete routing-instance RI3 protocols bgp cluster
delete routing-instances RI3 protocols bgp group RI2 no-client-reflect
set routing-instances RI3 protocols bgp group RI4 cluster 3.3.3.3
admin@MX> show route advertising-protocol bgp 34.34.34.3
RI4.inet.0: 45 destinations, 46 routes (41 active, 4 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.4.1.0/24 Self 100 I
* 10.4.2.0/24 Self 100 I
* 10.4.3.0/24 Self 100 I
* 10.4.4.0/24 Self 100 I
* 34.34.34.0/24 Self 100 I
* 172.19.100.0/24 Self 100 I
* 172.19.101.0/24 Self 100 I
* 172.19.102.0/24 Self 100 I
* 172.19.103.0/24 Self 100 I
* 192.168.112.0/24 Self 100 I
* 192.168.113.0/24 Self 100 I
* 192.168.114.0/24 Self 100 I
* 192.168.115.0/24 Self 100 I
admin@MX> show route advertising-protocol bgp 23.23.23.2
RI3.inet.0: 55 destinations, 59 routes (55 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.3.1.0/24 Self 100 I
* 10.3.2.0/24 Self 100 I
* 10.3.3.0/24 Self 100 I
* 10.3.4.0/24 Self 100 I
* 10.4.1.0/24 34.34.34.4 100 I
* 10.4.2.0/24 34.34.34.4 100 I
* 10.4.3.0/24 34.34.34.4 100 I
* 10.4.4.0/24 34.34.34.4 100 I
* 23.23.23.0/24 Self 100 I
* 34.34.34.0/24 Self 100 I
* 172.18.100.0/24 Self 100 I
* 172.18.101.0/24 Self 100 I
* 172.18.102.0/24 Self 100 I
* 172.18.103.0/24 Self 100 I
* 172.19.100.0/24 34.34.34.4 100 I
* 172.19.101.0/24 34.34.34.4 100 I
* 172.19.102.0/24 34.34.34.4 100 I
* 172.19.103.0/24 34.34.34.4 100 I
* 192.168.108.0/24 Self 100 I
* 192.168.109.0/24 Self 100 I
* 192.168.110.0/24 Self 100 I
* 192.168.111.0/24 Self 100 I
* 192.168.114.0/24 34.34.34.4 100 I
* 192.168.115.0/24 34.34.34.4 100 I
admin@MX> show route advertising-protocol bgp 12.12.12.1
RI2.inet.0: 69 destinations, 73 routes (69 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.2.1.0/24 Self 100 I
* 10.2.2.0/24 Self 100 I
* 10.2.3.0/24 Self 100 I
* 10.2.4.0/24 Self 100 I
* 12.12.12.0/24 Self 100 I
* 23.23.23.0/24 Self 100 I
* 172.17.100.0/24 Self 100 I
* 172.17.101.0/24 Self 100 I
* 172.17.102.0/24 Self 100 I
* 172.17.103.0/24 Self 100 I
* 192.168.104.0/24 Self 100 I
* 192.168.105.0/24 Self 100 I
* 192.168.106.0/24 Self 100 I
* 192.168.107.0/24 Self 100 I
admin@MX> show route advertising-protocol bgp 23.23.23.3
RI2.inet.0: 69 destinations, 87 routes (69 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.2.1.0/24 Self 100 I
* 10.2.2.0/24 Self 100 I
* 10.2.3.0/24 Self 100 I
* 10.2.4.0/24 Self 100 I
* 12.12.12.0/24 Self 100 I
* 23.23.23.0/24 Self 100 I
* 172.17.100.0/24 Self 100 I
* 172.17.101.0/24 Self 100 I
* 172.17.102.0/24 Self 100 I
* 172.17.103.0/24 Self 100 I
* 192.168.104.0/24 Self 100 I
* 192.168.105.0/24 Self 100 I
* 192.168.106.0/24 Self 100 I
* 192.168.107.0/24 Self 100 I
admin@MX> show route advertising-protocol bgp 34.34.34.4
RI3.inet.0: 55 destinations, 59 routes (55 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.2.1.0/24 23.23.23.2 100 I
* 10.2.2.0/24 23.23.23.2 100 I
* 10.2.3.0/24 23.23.23.2 100 I
* 10.2.4.0/24 23.23.23.2 100 I
* 10.3.1.0/24 Self 100 I
* 10.3.2.0/24 Self 100 I
* 10.3.3.0/24 Self 100 I
* 10.3.4.0/24 Self 100 I
* 12.12.12.0/24 23.23.23.2 100 I
* 23.23.23.0/24 Self 100 I
* 34.34.34.0/24 Self 100 I
* 172.17.100.0/24 23.23.23.2 100 I
* 172.17.101.0/24 23.23.23.2 100 I
* 172.17.102.0/24 23.23.23.2 100 I
* 172.17.103.0/24 23.23.23.2 100 I
* 172.18.100.0/24 Self 100 I
* 172.18.101.0/24 Self 100 I
* 172.18.102.0/24 Self 100 I
* 172.18.103.0/24 Self 100 I
* 192.168.104.0/24 23.23.23.2 100 I
* 192.168.105.0/24 23.23.23.2 100 I
* 192.168.106.0/24 23.23.23.2 100 I
* 192.168.107.0/24 23.23.23.2 100 I
* 192.168.108.0/24 Self 100 I
* 192.168.109.0/24 Self 100 I
* 192.168.110.0/24 Self 100 I
* 192.168.111.0/24 Self 100 I