Reads data from JSON files, supporting both single files and directories containing multiple JSON files. Supports JSONL (JSON Lines) format where each line is a separate JSON object.
JsonInput reads data from JSON files, supporting both single files and directories containing multiple JSON files. Supports JSONL (JSON Lines) format where each line is a separate JSON object. Can't specify both a single file path (ending with .json) and file_names parameter.
JsonInput inherits from the base Input class and provides specialized functionality for reading JSON and JSONL formatted data.
JsonInput
Classapplication_sdk.inputs.jsonInputReads data from JSON files, supporting both single files and directories containing multiple JSON files. Supports JSONL (JSON Lines) format where each line is a separate JSON object. Can't specify both a single file path (ending with .json) and file_names parameter.
Methods5
__init__
__init__(self, path: str, file_names: Optional[List[str]] = None, chunk_size: int = 100000)Parameters
pathstrfile_namesOptional[List[str]]chunk_sizeintget_dataframe
async get_dataframe(self) -> pd.DataFrameReturns
pd.DataFrame - Combined DataFrame from all specified JSON filesget_batched_dataframe
async get_batched_dataframe(self) -> AsyncIterator[pd.DataFrame]Returns
AsyncIterator[pd.DataFrame] - Iterator yielding batches of rowsget_daft_dataframe
async get_daft_dataframe(self) -> daft.DataFrameReturns
daft.DataFrame - Combined daft DataFrame from all specified filesget_batched_daft_dataframe
async get_batched_daft_dataframe(self) -> AsyncIterator[daft.DataFrame]Returns
AsyncIterator[daft.DataFrame] - Iterator yielding batches as daft DataFramesUsage Examples
Single file
Read a single JSON file
from application_sdk.inputs import JsonInput
json_input = JsonInput(path="data/users.json")
df = await json_input.get_dataframe()
Directory with all files
Read all JSON files from a directory
json_input = JsonInput(
path="s3://bucket/data/",
chunk_size=100000
)
async for batch_df in json_input.get_batched_dataframe():
# Process each batch
pass
See also
- Inputs: Overview of all input classes and common usage patterns
- SQLQueryInput: Read data from SQL databases by executing SQL queries
- ParquetInput: Read data from Parquet files, supporting single files and directories
- Application SDK README: Overview of the Application SDK and its components