The histograms and percentile estimates may have an error of up to 0.01% relative to the total number of rows. The frequent value counts may have an error of up to 0.01% when the number of distinct values is greater than 10000. The number of distinct values for categorical columns may have ~5% relative error for high-cardinality columns. When precise is set to false (the default), some returned statistics include approximations to reduce run time. UpdateMount(source: String, mountPoint: String, encryptionType: String = "", owner: String = null, extraConfigs: Map = Map.empty): boolean -> Similar to mount(), but updates an existing mount point instead of creating a new one Unmount(mountPoint: String): boolean -> Deletes a DBFS mount point RefreshMounts: boolean -> Forces all machines in this cluster to refresh their mount cache, ensuring they receive the most recent information Mounts: Seq -> Displays information about what is mounted within DBFS Mount(source: String, mountPoint: String, encryptionType: String = "", owner: String = null, extraConfigs: Map = Map.empty): boolean -> Mounts the given source directory into DBFS at the given mount point Rm(dir: String, recurse: boolean = false): boolean -> Removes a file or directory Put(file: String, contents: String, overwrite: boolean = false): boolean -> Writes the given String out to a file, encoded in UTF-8 Mv(from: String, to: String, recurse: boolean = false): boolean -> Moves a file or directory, possibly across FileSystems Mkdirs(dir: String): boolean -> Creates the given directory if it does not exist, also creating any necessary parent directories Ls(dir: String): Seq -> Lists the contents of a directory Head(file: String, maxBytes: int = 65536): String -> Returns up to the first 'maxBytes' bytes of the given file as a String encoded in UTF-8 For example, "%fs head -maxBytes=10000 /file/path" translates into "dbutils.fs.head("/file/path", maxBytes = 10000)".Ĭp(from: String, to: String, recurse: boolean = false): boolean -> Copies a file or directory, possibly across FileSystems The %fs shorthand maps straightforwardly onto dbutils calls. In notebooks, you can also use the %fs shorthand to access DBFS. For more info about a method, use dbutils.fs.help("methodName"). Most methods in this package can take either a DBFS path (e.g., "/foo" or "dbfs:/foo"), or another FileSystem URI. Dbutils.fs provides utilities for working with FileSystems.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |