In this section, we will guide you through the process of setting up Apache Kafka on your local machine. By the end of this module, you will have a working Kafka environment that you can use to practice and experiment with Kafka concepts.

Before we begin, ensure you have the following prerequisites:

• Java Development Kit (JDK) 8 or higher installed.
• Apache Kafka binary distribution downloaded.
• Basic understanding of command-line operations.

Kafka requires Java to run. Follow these steps to install the JDK:

For Windows:

1. Download the JDK from the Oracle website.
2. Run the installer and follow the on-screen instructions.
3. Set the JAVA_HOME environment variable:
   • Open the Control Panel and go to System and Security > System > Advanced system settings.
   • Click on Environment Variables.
   • Under System variables, click New and add JAVA_HOME with the path to your JDK installation (e.g., C:\Program Files\Java\jdk-11.0.10).

For macOS:

1. Install Homebrew if you haven't already:

   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
   

2. Install the JDK using Homebrew:

   brew install openjdk@11
   

3. Add the JDK to your PATH:

   echo 'export PATH="/usr/local/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc
   source ~/.zshrc
   

For Linux:

1. Install the JDK using your package manager. For example, on Ubuntu:

   sudo apt update
   sudo apt install openjdk-11-jdk
   

1. Download the latest Kafka binary distribution from the Apache Kafka website.
2. Extract the downloaded archive to a directory of your choice. For example:

   tar -xzf kafka_2.13-2.8.0.tgz
   cd kafka_2.13-2.8.0
   

Kafka uses ZooKeeper to manage distributed brokers. Start the ZooKeeper server using the following command:

You should see logs indicating that the ZooKeeper server has started successfully.

In a new terminal window, navigate to the Kafka directory and start the Kafka server:

You should see logs indicating that the Kafka server has started successfully.

Kafka topics are used to organize and store messages. Create a new topic named test:

You should see a confirmation message indicating that the topic has been created.

Open a new terminal window and start a Kafka producer to send messages to the test topic:

Type a few messages and press Enter after each message. The messages will be sent to the Kafka topic.

Open another terminal window and start a Kafka consumer to read messages from the test topic:

You should see the messages you produced in the previous step.

In this section, you have successfully set up a local Kafka environment. You installed the necessary prerequisites, started the ZooKeeper and Kafka servers, created a Kafka topic, and produced and consumed messages. This setup will serve as the foundation for exploring Kafka's core concepts and operations in the subsequent modules.

Next, we will dive into Kafka's core concepts, starting with producers and consumers.