Bug 6891 - Add support of lz4 (de)compression on MS Windows
Summary: Add support of lz4 (de)compression on MS Windows
Status: RESOLVED NEXTRELEASE
Alias: None
Product: SQL
Classification: Unclassified
Component: all (show other bugs)
Version: 11.37.7 (Jun2020)
Hardware: All Windows
: Normal enhancement
Assignee: SQL devs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-11 15:41 CEST by Martin van Dinther
Modified: 2020-10-21 14:41 CEST (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin van Dinther cwiconfidential 2020-06-11 15:41:24 CEST
User-Agent:       Mozilla/5.0 (Windows NT 6.1; rv:74.0) Gecko/20100101 Firefox/74.0
Build Identifier: 

Export of data to a file with extension .lz4 is currently not possible on Windows release. It returns an error.
Simularly files compressed with lz4 and extension .lz4 cannot be read as a decompressed csv/tsv file.

Add support for lz4 (de)compression on Windows by including liblz4.dll at installation (like done for libbz2.dll, zlib1.dll).
Sources and/or Binaries (lz4_win##_v1_9_2.zip) kan be found at https://github.com/lz4/lz4/releases/tag/v1.9.2

Reproducible: Always

Steps to Reproduce:
COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.gz';
-- succeeds
COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.lz4';
-- returns error

CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.gz');
-- succeeds
CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.lz4');
-- returns error
Actual Results:  
sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.gz';
271 affected rows
sql>-- succeeds
sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.lz4';
could not open file 'D:\MonetDB\Exports\tables.tsv.lz4': No such file or directory
sql>-- returns error
sql>
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.gz');
sql>-- succeeds
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.lz4');
GDK reported error: store_hot_snapshot: Failed to open D:\MonetDB\Backups\demodb.tar..tmp..lz4 for writing
sql>-- returns error
sql>


Expected Results:  
sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.gz';
271 affected rows
sql>-- succeeds
sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.lz4';
271 affected rows
sql>
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.gz');
sql>-- succeeds
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.lz4');
sql>-- succeeds
Comment 1 Martin van Dinther cwiconfidential 2020-10-14 18:07:36 CEST
This has been successfully implemented in MonetDB v11.39.5 (Oct2020).

Related hg change: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fad7c1e666c8