معرفی شرکت ها
databricks-connect-9.1.38
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
Databricks Connect Client
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | databricks-connect-9.1.38 |
| نام | databricks-connect |
| نسخه کتابخانه | 9.1.38 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Databricks |
| ایمیل نویسنده | feedback@databricks.com |
| آدرس صفحه اصلی | - |
| آدرس اینترنتی | https://pypi.org/project/databricks-connect/ |
| مجوز | Databricks Proprietary License |
# Databricks Connect
Databricks Connect is a Python library to run PySpark DataFrame queries on a remote Spark cluster.
Databricks Connect leverages the power of [Spark Connect].
An application using Databricks Connect runs locally, and when the results of a DataFrame query
need to be evaluated, the query is run on a configured Databricks cluster.
The following is a simple Python code that uses Databricks Connect and prints out a number range.
The number range query is executed on the Databricks cluster.
```python
from databricks.connect import DatabricksSession
session = DatabricksSession.builder.getOrCreate()
df = session.range(1, 10)
df.show()
```
## Specifying Connection Parameters
`DatabricksSession` offers a few ways to specify the Databricks workspace, cluster and user
credentials, collectively referred to in the rest of this document as connection parameters.
The specified credentials are used to execute the DataFrame queries on the cluster. This user must
have cluster access permissions and appropriate data access permissions.
*NOTE:* Currently, Databricks Connect only supports credentials based on [Personal Access
Token](https://docs.databricks.com/administration-guide/access-control/tokens.html). Other
authentication mechanisms are coming soon.
When `DatabricksSession` is initialized with no additional parameters as below, connection
parameters are picked up from the environment.
```python
session = DatabricksSession.builder.getOrCreate()
```
First, the `SPARK_REMOTE` environment variable is used if it's configured.
If configured, the `SPARK_REMOTE` environment variable must contain the spark connect connection
string. Read more about spark connect [connection string].
```sh
SPARK_REMOTE="sc://<databricks workspace url>:443/;token=<bearer token>;x-databricks-cluster-id=<cluster id>"
```
If this environment variable is not configured, Databricks Connect will now look for connection
parameters using the [Databricks SDK].
The Databricks Python SDK reads these values from two locations - first from environment variables
that may be configured. For parameters not configured via environment variables, the 'DEFAULT'
profile, if set up, from the configuration file `.databrickscfg`.
The details on the environment variable and configuration file can be found in the [Databricks SDK].
> Similar to the authentication environment variables, the Databricks SDK reads the cluster
> identifier from the environment variable `DATABRICKS_CLUSTER_ID` or from the `cluster_id` entry
> in the config file.
When the defaults should not be used, the Databricks Connect session can be initialized explicitly
with a `Config` object from the Databricks SDK.
In the below example, we are configuring Databricks Session to use the `foo-user` profile from the
configuration file.
Read more on profiles in configuration files in the [Databricks SDK].
```python
from databricks.sdk.core import Config
from databricks.connect import DatabricksSession
config = Config(
profile="foo-user",
# ...
)
session = DatabricksSession.builder.sdkConfig(config).getOrCreate()
```
Connection parameters can also be specified directly in code.
```python
session = DatabricksSession.builder.remote(
host="<databricks workspace url>",
cluster_id="<databricks cluster id>",
token="<bearer token>"
).getOrCreate()
```
The spark connect [connection string] can also be specified directly in code.
```python
session = DatabricksSession.builder\
.remote("sc://<databricks workspace url>:443/;token=<bearer token>;x-databricks-cluster-id=<cluster id>")\
.getOrCreate()
```
In summary, connection parameters are collected in the following order. When all connection
parameters are available, evaluation is stopped.
1. Specified directly using `remote()`, either as a connection string or as keyword arguments.
2. Specified via the Databricks SDK using `sdkConfig()`.
3. Specified in the `SPARK_REMOTE` environment variable.
4. Specified via the [Databricks SDK]'s default authentication.
[Spark Connect]: https://www.databricks.com/blog/2022/07/07/introducing-spark-connect-the-power-of-apache-spark-everywhere.html
[connection string]: https://github.com/apache/spark/blob/master/connector/connect/docs/client-connection-string.md
[Databricks SDK]: https://pypi.org/project/databricks-sdk/
نیازمندی
| مقدار | نام |
|---|---|
| ==0.10.9.7 | py4j |
| - | six |
| >=1.0.5 | pandas |
| >=1.0.0 | pyarrow |
| >=1.48.1 | grpcio |
| >=1.48.1 | grpcio-status |
| >=1.56.4 | googleapis-common-protos |
| >=1.15 | numpy |
| ==0.0.7 | databricks-sdk |
| >=1.0.5 | pandas |
| >=1.0.0 | pyarrow |
| >=1.48.1 | grpcio |
| >=1.48.1 | grpcio-status |
| >=1.56.4 | googleapis-common-protos |
| >=1.15 | numpy |
| >=1.15 | numpy |
| >=1.15 | numpy |
| >=1.0.5 | pandas |
| >=1.0.0 | pyarrow |
| >=1.15 | numpy |
| >=1.0.5 | pandas |
| >=1.0.0 | pyarrow |
| >=1.15 | numpy |
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3.7 | Python |
نحوه نصب
نصب پکیج whl databricks-connect-9.1.38:
pip install databricks-connect-9.1.38.whl
نصب پکیج tar.gz databricks-connect-9.1.38:
pip install databricks-connect-9.1.38.tar.gz