Comment by Marek Grzenkowicz on Data Loss Prevention finds superfluous...
@SerhiiRohoza I was unsure where I should post my question. I have been using GCP extensively since May and it's only very recently I found the tracker. It's not advertised much and it is not indexed...
View ArticleComment by Marek Grzenkowicz on spacy NER does not recognize some numeric texts
This does not answer your question, but do you really need a NER for numeric values? Why not use regular expressions or pattern matching if you prefer to stick to spaCy?
View ArticleComment by Marek Grzenkowicz on How to limit rows in the table ? How to...
Limiting number of rows is not possible out-of-the-box. You will need to implement the logic for row ordering and purging yourself; then you can run it automatically with an INSERT trigger.
View ArticleComment by Marek Grzenkowicz on Why cant I extract a single column using pandas?
You can load Excel data into a DF using pandas.read_excel.
View ArticleComment by Marek Grzenkowicz on View - select all fields of two tables when...
If you create a view with SELECT *, columns added to the underlying tables later will not appear in the view automatically. See stackoverflow.com/q/22716598/95 for more information.
View ArticleComment by Marek Grzenkowicz on How to Install virtualenv without pip?
I don't think you should be using pyenv script by referencing them directly (e.g. /home/yuis/.pyenv/shims/pip). What does pyenv versions show?
View ArticleComment by Marek Grzenkowicz on Writing huge Pandas HD5 dataframe to disk
Does this answer your question? OverflowError while saving large Pandas df to hdf
View ArticleComment by Marek Grzenkowicz on How to parameterize the config file for eksctl?
I ended up using envsubst but the idea is the same, so I accepted your answer.
View ArticleComment by Marek Grzenkowicz on Is the CI_JOB_TOKEN really not authorized to...
Same problem here. I hoped to use CI_JOB_TOKEN, but I will need to use a project access token instead.
View ArticleComment by Marek Grzenkowicz on Pytest capture not working - caplog and...
@Stew It's PyCharm, Variables pane in debugging mode I think.
View ArticleComment by Marek Grzenkowicz on SageMaker Estimator use_spot_instances causes...
max_wait= 23 * 60 * 60 and max_run = 24 * 60 * 60 - is should be the other way around, right? max_wait` should be greater than max_run.
View ArticleComment by Marek Grzenkowicz on virtualenv creation fails with error...
@KarlKnechtel In CI/CD logs, I had a stack trace from pre-commit trying to create its virtual environment and this led me to base.py#L120 where I discovered entry_points.txt being loaded. But in the...
View ArticleComment by Marek Grzenkowicz on How to enable command completion for Azure...
Path for Ubuntu: /home/linuxbrew/.linuxbrew/etc/bash_completion.d/az
View ArticleAnswer by Marek Grzenkowicz for How do I assign the output of a command to a...
Try:number_of_lines = !wc -l < cord_19.json!echo $number_of_linesor:number_of_lines = !wc -l < cord_19.jsonprint(number_of_lines)See Pipe Ipython magic output to a variable? for related discussion.
View ArticlePytest capture not working - caplog and capsys are empty
I am trying to create unit tests for a logger, but - even though the events are logged - the caplog and capsys remain empty.What am I doing wrong?
View ArticleAnswer by Marek Grzenkowicz for Is it possible to create a temporary table in...
No, a view consists of a single SELECT statement. You cannot create or drop tables in a view.Maybe a common table expression (CTE) can solve your problem. CTEs are temporary result sets that are...
View ArticleAnswer by Marek Grzenkowicz for How to cache pip packages within Azure Pipelines
I used the pre-commit documentation as inspiration:https://pre-commit.com/#azure-pipelines-examplehttps://github.com/asottile/azure-pipeline-templates/blob/master/job--pre-commit.ymland configured the...
View ArticlefileOffset empty in Data Loss Prevention results for PDF and DOCX files
I configured a DLP inspection job using GCP Console to scan PDF and DOCX files. It is working as expected, finding the expected entities and saving results to a BigQuery table.According to the docs,...
View ArticleAnswer by Marek Grzenkowicz for How to preserve line breaks in text content...
The issue had nothing to do with DLP.I was sending invalid JSON:{"item": {"value": "Eleanor Rigby Pharmacist Liverpool Hospital eleanor.rigby@example.com" },"deidentifyConfig": { ... }}Replacing the...
View ArticleHow to preserve line breaks in text content de-identified with Data Loss...
I am using an API call content.deidentify to de-identify text content. It is working as expected, but newline characters get stripped.API callcurl -s \ -H "Authorization: Bearer $(gcloud auth...
View ArticleAnswer by Marek Grzenkowicz for Directed probability graph - algorithm to...
I found this question while researching directed cyclic graphs. The probability of reaching each of the final nodes can be calculated using absorbing Markov chains.The video Markov Chains - Part 7 (+...
View ArticleAnswer by Marek Grzenkowicz for Is there a predefined variable for...
No, there is no such variable. You can use one of the /home/vsts/work paths and cache $(System.WorkFolder)/../.cache/torch/.Values of predefined variables for an Ubuntu agent:Agent.AcceptTeeEula True...
View ArticleIs there a predefined variable for "/home/vsts" in Azure Pipelines?
I need to cache files in /home/vsts/.cache/torch/.Is there a predefined variable for the home folder?
View ArticleAnswer by Marek Grzenkowicz for Is there a limit to how long a SQL query can...
I never encountered a problem of SQL query being too long in terms of number of characters, but there is a maximum number of tables a query can reference (256) and I hit this limitation a few...
View Articlecannot connect to Heroku Postgres - error "server closed the connection...
I am trying to use the Heroku CLI to connect a Postgres database in the cloud.heroku pg:info shows the database information, but heroku pg:psql fails with the following error:$ heroku pg:psql-->...
View ArticleFunction key reset by Azure?
I have just finished fixing a function (an HTTP trigger) that stopped working due to the deployment of Azure Functions Runtime 2.0.12050-alpha. The error shown in the portal was The binding type(s)...
View ArticleDocker image with 2 different SHA256 digests
What does it mean when inspect shows two RepoDigests values for a Docker image?Example:$ docker inspect aquasec/trivy:latest[ {"Id":...
View ArticleHave you ever encountered a query that SQL Server could not execute because...
Have you ever seen any of there error messages?-- SQL Server 2000Could not allocate ancillary table for view or function resolution.The maximum number of tables in a query (256) was exceeded.-- SQL...
View ArticleAnswer by Marek Grzenkowicz for DWH and ETL explained
This answer by no means should be treated as a complete definition of a data warehouse. It's only my attempt to explain the term in layman's terms.Transactional (operational, OLTP) and analytical (data...
View ArticleAnswer by Marek Grzenkowicz for How to remove the time portion of a datetime...
Itzik Ben-Gan in DATETIME Calculations, Part 1 (SQL Server Magazine, February 2007) shows three methods of performing such a conversion (slowest to fastest; the difference between second and third...
View ArticleAnswer by Marek Grzenkowicz for RDLC Switch Construct - is there an Else?
The Switch function returns the value associated with the first expression in a series that evaluates to true, you can use the following trick:= Switch( Fields!RATE_CODE.Value = "First" , " £/Week",...
View ArticleDask clients running in Kubeflow cannot communicate
I am using to following snippet to start an ephemeral Dask cluster within a node of a Kubeflow pipeline:from dask_kubernetes import KubeClusterfrom distributed import Clientcluster =...
View ArticleResources available locally but "No such file or directory" when packaged and...
I use paths defined as constants, e.g. TF_CONSTS = 'consts/tf_keras_param_config.json' and they work fine during development.However, when the package is built with Poetry (i.e. poetry build --format...
View ArticleAnswer by Marek Grzenkowicz for Resources available locally but "No such file...
Relative paths work fine during development, but - when the code is packaged and then installed - they must be translated to absolute ones and these must be determined dynamically (they depend on the...
View ArticleStart execution of existing SageMaker pipeline using Python SDK
SageMaker documentatin explains how to run a pipeline, but it assumes I have just defined it and I have the object pipeline available.How can I run an existing pipeline with Python SDK?I know how to...
View ArticleAnswer by Marek Grzenkowicz for virtualenv creation fails with error...
The issue is caused by the removal of entry_points.txt from site-packages/virtualenv-20.17.0.dist-info/.This files defines entry points, including console_scripts ones that allow the shell to call...
View ArticleAnswer by Marek Grzenkowicz for Restore database from .bak file with new name
In short:set the Restore.Database property to a new name,set the Restore.ReplaceDatabase property to true,specify new data and log files using the Restore.RelocateFiles property.
View ArticleHow to parameterize the config file for eksctl?
I have the following config file (edited for brevity):apiVersion: eksctl.io/v1alpha5kind: ClusterConfigmetadata: name: FOO-CLUSTER region: eu-central-1nodeGroups: - name: FOO-CLUSTER-group-1 [..] -...
View ArticleHow can I call a .NET DLL from an Inno Setup script?
I want to call a function from a .NET DLL (coded in C#) from an Inno Setup script.I have:marked the Register for COM interop option in the project properties,changed the ComVisible setting in the...
View ArticleCan I configure Visual Studio NOT to change StartUp Project every time I open...
Let's say that there is a solution that contains two projects (Project1 and Project2).Project1 is set as a StartUp Project (its name is displayed in a bold font). I double-click some file in Project2...
View ArticleComment by Marek Grzenkowicz on API management URL is giving Missing...
I had to send this key, but with a different header name - api-key.
View Article
