for what sins?
Dear Coffee God,
please
forgive me
your most zealous servant,
Kamil
inspirations
- two times pour coffee in the office on the floor and on my trausers
challenges
- not pour coffee 3th time
achievements
- close office & finished beta version of toolsets & best practices for new team, I waiting for feedback 🤞
- close office as the last person leave office, before 9PM
learnings
- pyspark: StructType, StructField, StringType
- convetion similar to result version from alembic
## pyspark
from pyspark.sql.types import StructType, StructField, StringType, DoubleType, IntegerType
schema = StructType([
StructField('country', StringType(), nullable=True),
StructField('time', DoubleType(), nullable=False),
StructField('year', IntegerType(), nullable=True),
])
# ...
# ### commands auto generated by Alembic - please adjust! ###
import sqlalchemy as sa
from alembic import op
def upgrade() -> None:
op.create_table(
"log",
sa.Column("country", sa.String(length=512), nullable=False),
sa.Column("time", sa.Float(), nullable=False),
sa.Column("status_code", sa.Integer(), nullable=False)
# ...
)
https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.sql.types.StructType.html
finds
- PDM
- modern Python package and dependency manager supporting the latest PEP standards. But it is more than a package manager
- it boosts your development workflow in various aspects.
- https://pdm-project.org/latest/
Thanks for reading this ❤️
YOUR most zealous servant,
Kamil