External Functions Tutorial | Snowflake смотреть онлайн

In this demo you'll learn how to expand the capabilities of your Snowflake data pipelines by easily extending them with external services and functionality.

To know more about how different companies are using External Function , you can refer this link --
https://www.vebuso.com/2021/04/bringing-extensibility-to-data-pipelines-whats-new-with-snowflake-external-functions/

Prerequisite:
Configuring a Snowflake Storage Integration to Access Amazon S3
https://youtu.be/Mje7AEpxsLA

Know about API Gateway in-depth--

What is a Proxy?
https://youtu.be/Fjexz_nbxiY
Concept of Reverse Proxy & API Gateway
https://youtu.be/X8VWVF-I7_4
Introduction to API Gateway with in-depth intuition
https://youtu.be/8lCX7CaJ4wM
Creating a POST Api | AWS API Gateway Passing Data to AWS Lambda
https://youtu.be/vgl9q5Ox5LE
Working With Query Parameters In API | API Gateway
https://youtu.be/JoR7ECrHDtg
Create a REST API on AWS | Step by Step Tutorial
https://youtu.be/yp4YggJG9nA
Usage Plan & API key - Amazon API Gateway
https://youtu.be/fxtHYQJt94A

Snowflake External Function Creation Steps:
-----------------------------------------------------------------------
Step 1: Creating Lambda Function
Step 2: Create Rest API
Step 3: Snowflake IAM Role


Demo 1:
--------------

import json

def lambda_handler(event, context):
# TODO implement
print(event)
translated=[]
for row in event['data']:
data=row[1]
message="Hello {}".format(data)
translated.append([row[0],message])
return {
'statusCode': 200,
'data': translated
}



Demo 2:
---------------
import json

def lambda_handler(event, context):
# TODO implement
print(event)
translated=[]
for row in event['data']:
name=row[1]
department=row[2]
message="{} works in {} Department".format(name, department)
translated.append([row[0],message])
return {
'statusCode': 200,
'data': translated
}


Snowflake Code:
---------------------------
--drop database if required
drop database if exists ramu;
--Create Database
create database if not exists ramu;
use ramu;


CREATE OR REPLACE api integration hellotesting
api_provider = aws_api_gateway
api_aws_role_arn = '{Put your AWS Role here}'
enabled = true
api_allowed_prefixes = ({API Invoke URL});




DESCRIBE integration hellotesting;

create or replace external function helloworld
(Name varchar,Department varchar)
returns variant
api_integration = hellotesting
as {API Invoke URL};

select Name,DEPARTMENT,helloworld(Name,DEPARTMENT) from "DEMO_DB"."PUBLIC"."EMPLOYEE";

Check this playlist for more AWS Projects in Big Data domain:
https://youtube.com/playlist?list=PLjfRmoYoxpNopPjdACgS5XTfdjyBcuGku