Pushing your PM2 logs from an EC2 machine to AWS CloudWatch requires a few crucial steps. In this article, we'll go through each step in detail, but before we begin, it's essential to understand that EC2 logs will not automatically be pushed to CloudWatch. To facilitate this process, a CloudWatch agent needs to be installed on your EC2 instance.

To begin with, ensure that the correct IAM permissions have been set up. Here's a template for the permissions required:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams"
      ],
      "Resource": ["*"]
    }
  ]
}

Alternatively, you can use “logs:*" for simplicity and add it to the existing group

To install the CloudWatch log agent, use the following command:

sudo yum install –y awslogs.

Note that this command is specific to Amazon Linux, and you may need to adjust it based on your Linux distribution.

Next, update your region in /etc/awslogs/awscli.conf. By default, it points to

us-east-1:

[plugins]
cwlogs = cwlogs
[default]
region = ap-southeast-1

To specify the logs to be tracked, edit /etc/awslogs/awslogs.conf. By default, this file tracks logs from /var/log/messages. To get logs from your specific files, change the configuration. For example:

[/var/log/Your-Chosen-Name/error.log]
datetime_format = %b %d %H:%M:%S
file = /var/log/Your-Chosen-Name/error.log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = Your-LogGroup-Name

Here, log_stream_name = {instance_id} signifies that the log stream will be named after the instance id of the EC2 instance sending the logs. The initial_position = start_of_file tells the agent to start reading from the beginning of the file. Lastly, log_group_name = Your-LogGroup-Name refers to the name of the log group on CloudWatch. If it doesn't already exist, CloudWatch will create it for you.

To send your PM2 logs to this new location, you'll need to modify the ecosystem.config.js file like so:

module.exports = {
  apps: [{
    name: "Your-App-Name",
    script: "Start-Up-File-Name",
    error_file: "/var/log/Your-Chosen-Name/error.log",
    out_file: "/var/log/Your-Chosen-Name/out.log",
    watch: true,
    env: {
      NODE_ENV: 'Your-ENV'
    }
  }]
};

Save this file and restart PM2. With these steps, your logs are now being saved in the new directory.

Finally, start the CloudWatch agent using sudo service awslogs start (or sudo systemctl start awslogsd if you're using Amazon Linux). To ensure the agent starts upon system reboot, run sudo systemctl enable awslogsd.service.

Now, login to the AWS console, navigate to CloudWatch, and check the 'logs' tab. Here, you should be able to find your log group and see your logs streaming from your EC2 instance. For more detailed instructions, check out the AWS documentation here.