The pipeline architecture, or pipes and filters architecture, is a commonly used monolithic architecture style in software design. As the name suggests, the topology of this architecture style consists of pipes and filters. Filters are self-contained nodes that perform a single task, pipes are one-way communication channels between filters.
Key Properties
Filter Properties
Filters are generally self-contained and stateless, and the tasks they perform generally have a single, well-defined goal. The four types of filters are:
- Producer: the starting point of a pipeline, which only has outbound pipes.
- Transformer: accepts inputs, performs a transformation on some or all of the data, and forwards it to the outbound pipe. This is known as the map stage of the MapReduce programming framework.
- Tester: accepts inputs, tests one or more criteria, and produces output based on the results of the test. This is known as the reduce stage of the MapReduce programming framework.
- Consumer: the terminal point of a pipeline, which only has inbound pipes, and sometimes persists the input data to a database or displays it on a user interface.
Pipe Properties
Pipes are generally unidirectional and point-to-point, meaning it does not broadcast from one filter to multiple filters. The payload carried on pipes are usually small, and they can be in any data format.
Trade-Offs
Advantages
- Great for simple, one-way processing because of the simplicity of this architecture style.
- This architecture style is extensible and modular, because new filters can be easily added, replaced, and deleted in the system.
Disadvantages
- Because it is a monolithic architecture style, deployments and testing are difficult. Even simple changes require the entire system to be fully tested and redeployed.
- Elasticity and scalability are low due to the monolithic nature of this architecture style. Although it is possible to scale up certain filters independently of others, it requires additional design techniques such as multithreading. However, applications can only scale to a certain point due to constraints of the monolithic processing and monolithic database.
- Fault tolerance is low because the monolithic design leads to a single point of failure. Any filter’s failure could lead to the failure of the entire pipeline.
