Yue Yin

Stream Processing (Stylus)

Work on data infrastructure of Alibaba Group. MaxCompute, DataHub MaxCompute (previously known as ODPS) is a general purpose, fully managed, multi-tenancy data processing platform for large-scale data warehousing. MaxCompute supports various data importing solutions and distributed computing models, enabling users to effectively query massive datasets, reduce production costs, and ensure data security. (https://www.alibabacloud.com/product/maxcompute) DataHub is a large scale distributed publish-subscribe(pub-sub) system developed for stream data processing. DataHub enables you to build applications based on stream data analysis, it can be also used as the data entrance of StreamCompute and MaxCompute. (https://data.aliyun.com/product/datahub) 2016-2017 : Work on metadata service in MaxCompute. Providing high capacity meta storage service with strong consistency, high availability and scalability to satisfy vary scenarios. 2015-2016: Work on DataHub project. Design and implement the in house storage and index sub-system for DataHub. Support billion level message read/write per second. End to End optimization, 5 times boost service QPS 2014-2015: Work on storage layer of MaxCompute. Participate in new column oriented file format design and implement. Our new file format supports nested data types, index, dynamic parameter tuning, run length encoding, dictionary encoding etc.

-Developing internal tools to speed up build & test process -Work with FTEs to develop new features in CTP ELT system, Increased system stability & availability, decreased data migration time and shorted report generation process