dremio 的InformationSchemaCatalog 服务一
InformationSchemaCatalog 服务实现了模仿information_schema的能力，让我们可以更好的在bi 工具以及其他系统使用
接口定义
/**
* Facet of the catalog service that provides metadata with an information_schema like API.
*
* The metadata provided may not satisfy the configured metadata policy requirement of various sources; the reads
* are directly from the KVStore without an inline refresh of the metadata.
*/
interface InformationSchemaCatalog {
/**
* List catalogs that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of catalogs
*/
Iterator listCatalogs(SearchQuery searchQuery);
/**
* List schemata that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of schemata
*/
Iterator listSchemata(SearchQuery searchQuery);
/**
* List tables (datasets) that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of tables
*/
Iterator
listTables(SearchQuery searchQuery);/**
* List views that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of views
*/
Iterator listViews(SearchQuery searchQuery);
/**
* List columns of all datasets that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of columns
*/
Iterator listTableSchemata(SearchQuery searchQuery);
}
子类
从上图可以看出InformationSchemaCatalog 的直接实现是没有包含PrivilegeCatalog，catalog 的子类需要实现此接口，具体的实现InformationSchemaCatalogImpl
核心依赖NamespaceService（此服务比较重要，dremio 利用了key,value 存储解决数据状态管理）
比如数据表获取的
@Override
public Iterator
listTables(SearchQuery searchQuery) {// 基于NamespaceService进行处理
final Iterable> searchResults =
userNamespace.find(getCondition(searchQuery));
return StreamSupport.stream(searchResults.spliterator(), false)
filter(IS_NOT_INTERNAL)
filter(IS_DATASET)
map(input -> {
final String sourceName = input.getKey().getRoot();
final TableType tableType;
if (input.getValue().getDataset().getType() == DatasetType.VIRTUAL_DATASET) {
tableType = TableType.VIEW;
else if ("sys".equals(sourceName) || "INFORMATION_SCHEMA".equals(sourceName)) {
// 包含了sys 以及INFORMATION_SCHEMA 的处理，sys 数据dremio 内部的，INFORMATION_SCHEMA 属于标准的INFORMATION_SCHEMA
tableType = TableType.SYSTEM_TABLE;
else {
tableType = TableType.TABLE;
}
return Table.newBuilder()
setCatalogName(DEFAULT_CATALOG_NAME)
setSchemaName(input.getKey().getParent().toUnescapedString())
setTableName(input.getKey().getName())
setTableType(tableType)
build();
})
iterator();
}
说明
InformationSchemaCatalog 的实现，完善了dremio 对于bi 以及jdbc 周边工具的集成，实现上不难，主要是利用了NamespaceService 服务
参考资料
sabot/kernel/src/main/java/com/dremio/exec/catalog/InformationSchemaCatalog.java https://docs.dremio.com/software/sql-reference/information-schema/tables/
版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。
暂时没有评论，来抢沙发吧~