S3 events notifications supports standard SNS topics and standard SQS queues as destinations but don’t support SNS FIFO and SQS FIFO.
S3 events notifications enables you to be notified whenever a specific event happens in your bucket. To receive the notification you must define the events you are interested in and the destination. The notifications are usually triggered in seconds but sometimes can take longer.
The events are –
New object creation
Object removal (versioned and non-versioned objects)
Object restore (e.g from Glacier)
Object lost on a reduced redundancy storage
The possible destinations include –
SQS – as mentioned above standard queues only and not FIFO queues
SNS- as mentioned above standard topics only and not FIFO topics
If when processing the events you write back to S3 be careful not to create an execution loop
While, SQS FIFO queues were introduced in 2016, SNS FIFO capabilities were introduced only on October 2020.
This capability is important for cases in which the order matters. E.g. bank transactions were you commit a transaction only if the balance remains non-negative.
Messages are grouped and ordered according to the message group ID. When sending a message you must specify a message group ID otherwise the action fails. If all the messages have the same message group ID then all the messages are sent and received in strict order. The message group id can be any value, e.g 12, “hello”, “user_id-123”, etc.
Note that as in the SQS case, the topic name must end with .fifo, a limitation that counts to the 80 characters restriction as well.
This means that all the messages which are pushed to this queue would be visible to the consumer after the delay period. The minimum delay which is also the default delay is 0 and the maximum is 15 minutes.
Note that when changing the delay of a queue the behaviour of FIFO queues and standard queues is different –
For standard queues, the per-queue delay setting is not retroactive—changing the setting doesn’t affect the delay of messages already in the queue.
For FIFO queues, the per-queue delay setting is retroactive—changing the setting affects the delay of messages already in the queue.
If you need to delay the visibility of specific messages and not all messages in the queue you can use message timers and add an initial invisibility period for a message. This is only supported by standard queues.Note that setting a message timer for individual messages overrides the delay period of the delay queue.
See the image below to understand message timeline in a queue –
Conducting a Successful Onboarding Plan and Onboarding Process – I believe that onboarding is important for the entire employment period. It helps setting expectations, getting to the code and being meaningful faster and assure both sides they made the right choice (and if not know it in an early stage). One thing I miss in this plan is the social part which I think is also important – having lunch \ coffee \ etc with not just the mentor. I look forward to the next part “Conducting a Successful Offboarding Plan and Offboarding Process”. It might sound like a joke, but it is not. Good offboarding process can help the organization learn and grow and leave the employee with a good taste so she might come back in the future or recommend her friends to join \ use the product.
The challenges of AWS Lambda in production – serverless is gaining popularity in the last years and specifically AWS lambda. While many times it sounds like a magic solution for scalability and isolation it also has its issues to know. In this post Lucas De Mitri from Sinch presents problems they run into and possible solutions. For a high level view on Lambda functions just read the conclusion part.
My Arsenal Of AWS Security Tools – In a preview post I pointed out on ElectricEye a tool to continuously monitor your AWS services for configurations that can lead to degradation of confidentiality, integrity or availability. This github repo aggregates open source tools for AWS security: defensive, offensive, auditing, DFIR, etc.
3 Problems to Stop Looking For in Code Reviews – I find the post title inaccurate but I like the attitude. As a reviewer you should not be bothered by tiny issues that can be enforced by tooling. Few tools are mentioned in the post and I would also add to that githooks which I find very powerful.I also agree with the insight that code reviews usually happen too late in the development process and constantly looking for the balance between letting developers progress and move forward and on the other hand give feedback on the right time.
The Power of Product Thinking – In a previous post I mentioned that understanding the cost structure and trade-offs between different architecture (cost wise but also performance and feature wise) is a way to become a more valuable team member. Product thinking is another skill that can make you a more valuable and influential team member. This post explains what product thinking is (and isn’t) and completes it by suggesting several practices on how to develop product thinking. Totally liked it and am going to adopt some of the suggested practices .