aligned.sources.databricks.UCTableSource

class documentation

class UCTableSource(CodableBatchDataSource, WritableFeatureSource, DatabricksSource): (source)

View In Hierarchy

A source that connects to a Databricks Unity Catalog table

Class Method multi_source_features_for Undocumented
Method all_between_dates Undocumented
Method all_data Undocumented
Async Method feature_view_code Setup the code needed to represent the data source as a feature view
Method features_for Undocumented
Async Method freshness .table("my_table") .freshness()
Async Method insert Undocumented
Method job_group_key A key defining which sources can be grouped together in one request.
Async Method overwrite Undocumented
Method overwrite_schema Undocumented
Async Method schema Returns the schema for the data source
Async Method upsert Undocumented
Method with_config Undocumented
Class Variable config Undocumented
Class Variable should_overwrite_schema Undocumented
Class Variable table Undocumented
Class Variable type_name Undocumented

Inherited from CodableBatchDataSource:

Property as_markdown Undocumented
Class Method _deserialize Undocumented
Method _serialize Undocumented

Inherited from BatchDataSource (via CodableBatchDataSource):

Method __hash__ Undocumented
Method all Undocumented
Method all_columns Undocumented
Method depends_on Undocumented
Method filter Undocumented
Method location_id Undocumented
Method needed_configs Undocumented
Method source_id An id that identifies a source from others.
Method tags Undocumented
Method transform_with_polars Undocumented
Method with_loaded_at Undocumented
Method with_view Undocumented
@classmethod
def multi_source_features_for(cls: type[UCTableSource], facts: RetrievalJob, requests: list[tuple[UCTableSource, RetrievalRequest]]) -> RetrievalJob: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.multi_source_features_for

Undocumented

def all_between_dates(self, request: RetrievalRequest, start_date: datetime, end_date: datetime) -> RetrievalJob: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.all_between_dates

Undocumented

def all_data(self, request: RetrievalRequest, limit: int | None) -> RetrievalJob: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.all_data

Undocumented

async def feature_view_code(self, view_name: str) -> str: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.feature_view_code

Setup the code needed to represent the data source as a feature view

```python FileSource.parquet("my_path.parquet").feature_view_code(view_name="my_view")

>>> """from aligned import FeatureView, String, Int64, Float

class MyView(FeatureView):

metadata = FeatureView.metadata_with(
name="Embarked", description="some description", batch_source=FileSource.parquest("my_path.parquet") stream_source=None,

)

Passenger_id = Int64() Survived = Int64() Pclass = Int64() Name = String() Sex = String() Age = Float() Sibsp = Int64() Parch = Int64() Ticket = String() Fare = Float() Cabin = String() Embarked = String()"""

```

Returns:
str: The code needed to setup a basic feature view
def features_for(self, facts: RetrievalJob, request: RetrievalRequest) -> RetrievalJob: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.features_for

Undocumented

async def freshness(self, feature: Feature) -> datetime | None: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.freshness

my_table_freshenss = await (PostgreSQLConfig("DB_URL")
.table("my_table") .freshness()

)

async def insert(self, job: RetrievalJob, request: RetrievalRequest): (source)
overrides aligned.feature_source.WritableFeatureSource.insert

Undocumented

def job_group_key(self) -> str: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.job_group_key

A key defining which sources can be grouped together in one request.

async def overwrite(self, job: RetrievalJob, request: RetrievalRequest): (source)
overrides aligned.feature_source.WritableFeatureSource.overwrite

Undocumented

def overwrite_schema(self, should_overwrite_schema: bool = True) -> UCTableSource: (source)

Undocumented

async def schema(self) -> dict[str, FeatureType]: (source)
overrides aligned.data_source.batch_data_source.BatchDataSource.schema

Returns the schema for the data source

`python source = FileSource.parquet_at('test_data/titanic.parquet') schema = await source.schema() >>> {'passenger_id': FeatureType(name='int64'), ...} `

Returns:
dict[str, FeatureType]: A dictionary containing the column name and the feature type
async def upsert(self, job: RetrievalJob, request: RetrievalRequest): (source)
overrides aligned.feature_source.WritableFeatureSource.upsert

Undocumented

def with_config(self, config: DatabricksConnectionConfig) -> UCTableSource: (source)

Undocumented

config: DatabricksConnectionConfig = (source)
overrides aligned.sources.databricks.DatabricksSource.config

Undocumented

should_overwrite_schema: bool = (source)

Undocumented

table: UnityCatalogTableConfig = (source)

Undocumented

type_name: str = (source)
overrides aligned.data_source.batch_data_source.CodableBatchDataSource.type_name

Undocumented