Updated March 21, 2023

Introduction to Talend Open Studio

Talend offers Open Studio, which is an open-source for data integration. It has more than 800+ components for various integration purposes. Download Talend Open Studio from https://www.talend.com/download/

Data integration means combining data from different sources and combining them to a single view to get some meaningful data from that which can help the company or organization to improve their business by analyzing those data. Integration helps get data, clean the data making some required transformation, etc., then load it into a data warehouse.

What is Talend?

Talend is an ETL tool that is used for data integration. Talend provides a solution for data preparation, data quality, data integration, and big data. Here we will be discussing some of the components. To make it easy to see the below example, A sim operator has huge data about plans, customers, sim details, etc. These data are huge, so big data is also used in the integration.

Customer A buying a sim using a government id

Giving his name as AB C

address as Chennai, Chennai

phone number as 1234567890

After data integration

First name: AB

Last name:C

Address: Chennai, India

Phone number:+911234567890

Here the data is cleansed and transformed into something more meaningful.

Benefits

The benefits are pointed out below:

Analyzing Business trends using data integration
Combining data into a single system
Time-saving and more efficient and less rework
Easy Report generation – used by BI tools
Maintaining and inserting data into the data warehouse and data marts

Application

Here are the following applications mention below

1. Working with Talend

Make sure you have java installed and environment variables set.
Download the open-source from the Talend website and install the software.
Create a new project and finish the setup
Talend will open with the designer tab.
Talend is an eclipse-based tool, and the components can be dragged from the palette, or you can click and type the component’s name.

2. The first job Reading a file

Search for the component tFileinputdelimited. This component is used for reading any delimited files.
Place the tfileinputdelimited component. Search for tlogrow and place it in the job designer.
Right-click tfileinputdelimited and select row-> main and draw a line to tlogrow.
In the component, the tab selects the path of the file you want to read and gives the row separator as \n. If the file has a delimiter, you can mention the delimiter.
Click the schema and give the column type details, or you can read the entire row as a string with one column, and the delimiter value should be empty.
You can skip the header and footer also.
In the tlogrow component, select the way how you want to see the data. Table format or single-line format.
tlogrow displays output in the run console.
After connecting both tfileinputdelimited and tlogrow, run the job from the run tab.
You can see the file contents in the console printed.

3. A second job using Tmap

Read a file and filter it into different output files.
Read a file in the tfileinputdelimited component with one column schema as the record.
Tmap component- This component helps in transforming data with some inbuilt functions like lookup, joins, etc.
In tmap, create two outputs out1 and out2.
In out1 filter add row3.record.contains(“talend”) and draw the record to out1.
Draw the record line to other out2.

From the tmap, take main rows and connect to two tfileoutputdelimited.
out1 link to one tfileoutputdelimited1 as file1.txt and out2 to other tfileoutputdelimited2 as file2.txt.
txt will have records that contain talend.
txt will have records that have other names.

4. Built-in and repository

Built-in means you should set schema or details for connecting to a database every time.
The repository comes in handy to save the details in the metadata so that you can reuse the same details every time without manually entering details every time. You can save file schema, database connections, Hadoop connection, hive connection, s3 connection, and many more in metadata.

Components of Talend Open Studio

Here are the following Components of Talend Open Studio mention below

1. tFileList

This component lists the files in a directory or folder with a given file mask pattern.

2. tMysqlConnection

This component is used for connecting with the mysql database.
Mysql components can use this connection for easy setup of connecting to the database.

3. tMysqlinput

This component helps to run a mysql database query and get the table or columns. This component is used to select queries and get the details.

4. tMysqlOutput

This component is used for inserting or updating data in the Mysql database.

5. tprejob

This component is the first to execute in the job and can be connected with other components with on subjob ok.

6. tpostjob

This component is the last to execute in the job. You can connect this with connection close components.

7. tlogcatcher

This component catches the warning and errors in the job.
The most important component is used in the error handling technique.
Error logs can be written using this component along with tfileoutputdelimited.
There are more than 800+ components.

Context variable

Context variables are variables that can be used in the job anywhere.
It holds values and can be passed to another job also using trun component.
The use of context variables is we can change the value for different purposes.
For example, we can have a set of values for the development context group and a different set of context values for production.
This way, we don’t have to change the job; just changing the context parameters is enough.

Building a job

Let us know how to select and build a job.

To build a job, right-click the job and select a build job.
You can import the build job in TAC.
In Talend Administration Console, you schedule a job to trigger the job set dependency also.
You can also import the job from the Nexus repository using an artifact job.

Create a task in TAC

Let us discuss how to create a task in TAC.

Open job conductor in TAC
Click new tasks and select normal or artifact tasks.
Import the builded job or select from nexus.
Select the job server in which the talend will run.
Save the task.
Now you can deploy and run the job.

Conclusion

“Simplify ETL and ELT with the leading free open source ETL tool for big data.” is the tagline for open studio. Talend Bigdata has many components for handling huge data. Standard job, Bigdata job, Bigdata streaming jobs are the different types of jobs available in Talend. Bigdata jobs can be created in a spark or MapReduce framework.

Quiz Result
Total Questions	Correct Answers	Wrong Answers	Percentage