You can see my reply at the end of this blog post:
I will repeat the steps I took to get this "working", however I think that I spoke too soon:
1. Clone the repo (git clone https://github.com/Juniper/vSRX-AWS.git && cd vSRX-AWS/Transit-VPC/)
2. Zip & Upload the `Configurator` and `Poller` lambda function files to a NEW S3 bucket (don't forget permissions!)
3. Modify `transit-vpc-primary-account.template` to update two 'CodeLocation' values with previously created S3 bucket
4. Accept vSRX EULA(s) in AWS Marketplace
5. Finally deploy the CF Stack using modified `transit-vpc-primary-account.template` JSON file
Once this is done, I verified that all of the AWS resources are created (instances, lamda functions, cloudwatch trigger, s3 bucket, etc.). After 15 minutes the poller script is invoked and it does discover the tagged spoke VPC VGWs and creates the VPNs within AWS, the configs are loaded into the S3 vpnconfigs/ folder, however it doesn't ever appear to be configuring the actual vSRX devices themselves. I tracked it down to the "configurator" script being excuted and seeing this error in CloudWatch:
START RequestId: 3cb2d938-7191-11e7-a5bb-f36ee4ed22e8 Version: $LATEST
Unable to import module 'transit-vpc-push-juniper-config': No module named paramiko
END RequestId: 3cb2d938-7191-11e7-a5bb-f36ee4ed22e8
REPORT RequestId: 3cb2d938-7191-11e7-a5bb-f36ee4ed22e8 Duration: 0.33 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 29 MB
Any ideas on this? I'm guessing it's "import paramiko" in transit-vpc-push-juniper-config.py
Try this for transit-vpc-push-juniper-config instead of uploading py file;
- create folder transit-vpc-push-juniper-config
- copy transit-vpc-push-juniper-config.py file under it
- copy paramiko module under it. This folder can be copied from download (exported) file from solution helper lambda.
- zip the folder with zip extension, upload to s3, and adjust the variable in Cloudformtion as such.
The CloudFormation template in git had the variable with zip, but the documentation didnt explain how to add paramiko and zip the folder before uploading to s3.
Thanks, I've done that as you've said. The configuration still doesn't seem to get applied, and I now see this in cloudwatch logs for the configuration function:
Unable to import module 'transit-vpc-push-juniper-config': No module named transit-vpc-push-juniper-config
I'm not an AWS Lambda expert but that seems strange, I don't know where and why it'd be trying to use that particular string as a Python package.
Just to note, I used the following guide to create the zip (which ended in the same result as what you instructed):
Also, it'd seem that if Juniper hosted public S3 buckets and then hard-coded those into CF templates that it would make this a tad easier for customers and for Juniper. For example, you wouldn't need to instruct customers on how to create these customized Lambda packages. Just a thought.
One potential issue with uploading the configurator zip file to your S3 bucket is if you zip the folder from outside the folder instead of creating the zip file from inside (i.e. do not include the folder when creating the zip file)
Btw, is there any reason why you would not have your template point to the default S3 bucket (which is the default value for the codeLocation) ?
I do agree that this makes the most sense, and the "juniper-transit-vpc/" bucket does seem accesible, for example here. However for some reason when the CF stack is invoked it's unable to retrieve and/or use those files to create the poller and configurator functions. Perhaps that's the root of the issue?
Just a quick update.
I downloaded the .zip found in the public Juniper bucket I posted above and then uploaded it to my own bucket for use, then reflected that in my CF stack. It appears to work as now all three lambda functions get created. I'm not sure why the Juniper public bucket doesnt work - perhaps it's only in a different region from where I'm trying to deploy.
Anyway the VPNs are being created successfully in the spoke VPCs however the vSRX appliances themselves still aren't being configured. Presumably there's a timeout issue, thought I don't know where or why, though I think I will increase the timeout from 300s to high just in-case. This is what I now see this in the cloudwatch logs:
START RequestId: 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Version: $LATEST
[INFO] 2017-07-27T19:30:59.940Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Found credentials in environment variables.
[INFO] 2017-07-27T19:31:00.861Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Downloading config file: https://s3-us-west-2.amazonaws.com/vsrxtvpctest-vpnconfigs3bucket-k6iy7iwkzfgw/vpnconfigs/transit_vp...
[INFO] 2017-07-27T19:31:00.922Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Starting new HTTPS connection (1): vsrxtvpctest-vpnconfigs3bucket-k6iy7iwkzfgw.s3-us-west-2.amazonaws.com
[INFO] 2017-07-27T19:32:01.981Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Starting new HTTPS connection (2): vsrxtvpctest-vpnconfigs3bucket-k6iy7iwkzfgw.s3-us-west-2.amazonaws.com
[INFO] 2017-07-27T19:33:02.520Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Starting new HTTPS connection (3): vsrxtvpctest-vpnconfigs3bucket-k6iy7iwkzfgw.s3-us-west-2.amazonaws.com
[INFO] 2017-07-27T19:34:06.149Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Starting new HTTPS connection (4): vsrxtvpctest-vpnconfigs3bucket-k6iy7iwkzfgw.s3-us-west-2.amazonaws.com
[INFO] 2017-07-27T19:35:13.145Z 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Starting new HTTPS connection (5): vsrxtvpctest-vpnconfigs3bucket-k6iy7iwkzfgw.s3-us-west-2.amazonaws.com
END RequestId: 1e6ba827-7302-11e7-adff-6fd4b4c8b833
REPORT RequestId: 1e6ba827-7302-11e7-adff-6fd4b4c8b833 Duration: 300001.79 ms Billed Duration: 300000 ms Memory Size: 128 MB Max Memory Used: 39 MB
I think the S3 bucket that you copied from has the old configurator python module which lacks some of the fixes that we later added. Can you please have your template point to the following S3 bucket/files instead?
Thanks for the reply.
Is this solution being tested by Juniper? I ask because I wonder why your implementation guide instructs users to create their own bucket for hosting these scripts when they seemingly exist on public S3 buckets. Also, these buckets don't seem to be accessible at the time of stack creation since the Lambda jobs fail to create. The message just seems a bit confusing.
Anyway, I downloaded the new file you mentioned (juniper-transit-vpc-solution/transit-vpc-push-juniper-config.zip) and, since it also doesn't seem accessible from us-east-1, have uploaded it to my own S3 bucket and was able to again successfully fully create the solution via CloudFormation.
The result however is still the same. Everything seems to work except for the vSRX actually being configured. CloudWatch logs show the same as what I posted yesterday:
REPORT RequestId: 16246b87-739b-11e7-b202-5dd0ead4480a Duration: 300003.41 ms Billed Duration: 300000 ms
This solution has already been tested by Juniper. However if you see any shortcomings or discrepancies in the guide/documentation we would be more than happy to work with you and address them.
Did you apply all the configurations documented in the guide to your deployed vSRXs? Without them, Juniper configurator's config push will fail.