wxSQLite3, sqlite3, static library and encryption Topic is solved

Talk here about issues with one of the components hosted at wxCode, or suggest features for it.
Post Reply
paulcrown
In need of some credit
In need of some credit
Posts: 8
Joined: Mon Jul 22, 2013 6:59 pm

wxSQLite3, sqlite3, static library and encryption

Post by paulcrown »

Greetings,

I am running Xubuntu 12.04 64bit. I have no trouble compiling sqlite3 and wxSQLite3 without encryption.

I have read numerous forums on the correct way to compile sqlite3 using the AES128 codec from wxSQLite3. A few refer to success, but the instructions are dead links. I have listed below what I have gleaned so far, but without success.
  • 1 Download sqlite3-3.7.9 and saved to directory sqlite3-3.7.9
    2 Download wxSQLite3-3.0.3 and saved to directory wxsqlite3-3.0.3
    3 Created a directory sqlite3-3.7.9/src/codec
    4 Copied the contents of wxsqlite3-3.0.3/sqlite3/secure/src/* to sqlite3-3.7.9/src/codec
    5 Added new objects to Makefile.in

Code: Select all

...
         vdbetrace.lo wal.lo walker.lo where.lo utf.lo vtab.lo \
         codec.lo codecext.lo rijndael.lo  
...
SRC = \
  $(TOP)/src/alter.c \
  $(TOP)/src/codec/codec.cpp \
  $(TOP)/src/codec/codecext.cpp \
  $(TOP)/src/codec/rijndael.cpp \
  $(TOP)/src/analyze.c \
...
alter.lo:	$(TOP)/src/alter.c $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/alter.c
	
codec.lo:	$(TOP)/src/codec/codec.cpp $(HDR)   
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/codec/codec.cpp
   
codecext.lo:	$(TOP)/src/codec/codecext.cpp $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/codec/codecext.cpp
   
rijndael.lo:	$(TOP)/src/codec/rijndael.cpp $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/codec/rijndael.cpp
	
analyze.lo:	$(TOP)/src/analyze.c $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/analyze.c
...
  • 6 Compiled sqlite with special contants defined

Code: Select all

CFLAGS="-DSQLITE_HAS_CODEC -DCODEC_TYPE=CODEC_TYPE_AES128 -DSQLITE_CORE -DTHREADSAFE=1 -DSQLITE_SECURE_DELETE -DSQLITE_SOUNDEX -DSQLITE_ENABLE_COLUMN_METADATA" ./configure --disable-shared --enable-static --disable-tcl --enable-threadsafe --enable-releasemode --disable-amalgamation && make
My first error relates to the fact that codec.c, codecext.c, and rijndael.c are .cpp files. Knowning that sometimes the compiler has problems treating them as such, I tried changing the Makefile.in above to .c, and when that was unsuccessful I left the Makefile.in with .cpp and just changed the file extention of the files in src/codec to .cpp. The following error occured either way.

Code: Select all

In file included from ./src/codec/codec.c:40:0:
./src/codec/codec.h:65:3: error: unknown type name 'Btree'
./src/codec/codec.h:66:24: error: 'SQLITE_MAX_PAGE_SIZE' undeclared here (not in a function)
./src/codec/codec.h:87:34: error: unknown type name 'Btree'
./src/codec/codec.h:92:1: error: unknown type name 'Btree'
./src/codec/codec.c: In function 'CodecInit':
./src/codec/codec.c:497:18: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
./src/codec/codec.c: At top level:
./src/codec/codec.c:526:29: error: unknown type name 'Btree'
./src/codec/codec.c:549:1: error: unknown type name 'Btree'
Can anyone with experience compiling wxSQLite3 and sqlite3 with encryption provide me with any assistance, either in what I am doing wrong, or provide me with a more accurate instruction set of how to compile with encryption?
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: wxSQLite3, sqlite3, static library and encryption

Post by utelle »

paulcrown wrote:I am running Xubuntu 12.04 64bit. I have no trouble compiling sqlite3 and wxSQLite3 without encryption.

I have read numerous forums on the correct way to compile sqlite3 using the AES128 codec from wxSQLite3. A few refer to success, but the instructions are dead links. I have listed below what I have gleaned so far, but without success.
  • 1 Download sqlite3-3.7.9 and saved to directory sqlite3-3.7.9
    2 Download wxSQLite3-3.0.3 and saved to directory wxsqlite3-3.0.3
    3 Created a directory sqlite3-3.7.9/src/codec
    4 Copied the contents of wxsqlite3-3.0.3/sqlite3/secure/src/* to sqlite3-3.7.9/src/codec
    5 Added new objects to Makefile.in
There is no need to download a separate SQLite3 source archive. The file release of wxSQLite3 3.0.3 contains all required source files.
paulcrown wrote:

Code: Select all

...
         vdbetrace.lo wal.lo walker.lo where.lo utf.lo vtab.lo \
         codec.lo codecext.lo rijndael.lo  
...
SRC = \
  $(TOP)/src/alter.c \
  $(TOP)/src/codec/codec.cpp \
  $(TOP)/src/codec/codecext.cpp \
  $(TOP)/src/codec/rijndael.cpp \
  $(TOP)/src/analyze.c \
...
alter.lo:	$(TOP)/src/alter.c $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/alter.c
	
codec.lo:	$(TOP)/src/codec/codec.cpp $(HDR)   
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/codec/codec.cpp
   
codecext.lo:	$(TOP)/src/codec/codecext.cpp $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/codec/codecext.cpp
   
rijndael.lo:	$(TOP)/src/codec/rijndael.cpp $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/codec/rijndael.cpp
	
analyze.lo:	$(TOP)/src/analyze.c $(HDR)
	$(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/analyze.c
...
  • 6 Compiled sqlite with special contants defined

Code: Select all

CFLAGS="-DSQLITE_HAS_CODEC -DCODEC_TYPE=CODEC_TYPE_AES128 -DSQLITE_CORE -DTHREADSAFE=1 -DSQLITE_SECURE_DELETE -DSQLITE_SOUNDEX -DSQLITE_ENABLE_COLUMN_METADATA" ./configure --disable-shared --enable-static --disable-tcl --enable-threadsafe --enable-releasemode --disable-amalgamation && make
My first error relates to the fact that codec.c, codecext.c, and rijndael.c are .cpp files. Knowning that sometimes the compiler has problems treating them as such, I tried changing the Makefile.in above to .c, and when that was unsuccessful I left the Makefile.in with .cpp and just changed the file extention of the files in src/codec to .cpp. The following error occured either way.
First of all, you should use the SQlite3 amalgamation source. This is the recommended SQlite3 source variant. Then, you obviously used an older wxSQLite3 file release. A long time ago the wxSQLite3 encryption extension was implemented in C++. Version 3.0.3 of wxSQLite3 comes with a pure C implementation.
paulcrown wrote:Can anyone with experience compiling wxSQLite3 and sqlite3 with encryption provide me with any assistance, either in what I am doing wrong, or provide me with a more accurate instruction set of how to compile with encryption?
Compiling SQLite3 with encryption support is just compiling the file sqlite3secure.c in subdirectory sqlite3/secure/src of the wxSQLite3 file release - all other required source files are included by this file. Following is the set of recommended/required compile time defines:

Code: Select all

    THREADSAFE=1
    SQLITE_SOUNDEX
    SQLITE_ENABLE_COLUMN_METADATA
    SQLITE_HAS_CODEC
    CODEC_TYPE=CODEC_TYPE_AES128
    SQLITE_SECURE_DELETE
    SQLITE_ENABLE_FTS3
    SQLITE_ENABLE_FTS3_PARENTHESIS
    SQLITE_ENABLE_RTREE
    SQLITE_CORE
    SQLITE_ENABLE_EXTFUNC
    SQLITE_USE_URI
I hope this helps a bit.

Regards,

Ulrich
paulcrown
In need of some credit
In need of some credit
Posts: 8
Joined: Mon Jul 22, 2013 6:59 pm

Re: wxSQLite3, sqlite3, static library and encryption

Post by paulcrown »

Thanks Ulrich :D That make much more sense.

I want to recap the steps I have taken:

I am running Xubuntu 12.04 LTS AMD64. I am going to use wxSQLite3-3.0.3 with wxWidgets-2.9.4.
  • 1 Download wxSQLite3-3.0.3 and saved to directory wxsqlite3-3.0.3

Code: Select all

wget http://downloads.sourceforge.net/project/wxcode/Components/wxSQLite3/wxsqlite3-3.0.3.tar.gz && tar xvfz wxsqlite*.tar.gz
cd wxsqlite3-3.0.3
  • 2 Compile sqlite3secure.c to use as my new sqlite3 library I took out -DSQLITE_ENABLE_EXTFUNC because of error - see below

Code: Select all

gcc -DTHREADSAFE=1 -DSQLITE_SOUNDEX -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_HAS_CODEC -DCODEC_TYPE=CODEC_TYPE_AES128 -DSQLITE_SECURE_DELETE -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_RTREE -DSQLITE_CORE -DSQLITE_USE_URI -c sqlite3/secure/src/sqlite3secure.c -o sqlite3secure.o && ar rcs lib/libsqlite3.a sqlite3secure.o && rm -f sqlite3secure.o
  • 3 Compile wxSQLite3 Note: I used configure29 when compiling because I am using wxWidgets-2.9

Code: Select all

CFLAGS="-DwxUSE_DYNAMIC_SQLITE3_LOAD=0 -DWXSQLITE3_HAVE_METADATA=1 -DWXSQLITE3_HAVE_CODEC=1 -DWXSQLITE3_HAVE_LOAD_EXTENSION=0" ./configure29 --enable-codec --enable-metadata --disable-shared --with-sqlite3-prefix=. && make
  • 4 Now I include -Iwxsqlite3-3.0.3/include and -Lwxsqlite3-3.0.3/lib -lwxcode_gtk2_wxsqlite3-2.9 in my project.
Encyption now works. Yeah!

Code: Select all

        if (wxSQLite3Database::HasEncryptionSupport()) {
          data.Open(database, pass);
        } else {
          data.Open(database);
        }
The error I had using -DSQLITE_ENABLE_EXTFUNC with the compile is:

Code: Select all

In file included from sqlite3/secure/src/sqlite3secure.c:58:0:
sqlite3/secure/src/extensionfunctions.c:213:25: error: conflicting types for 'i64'
sqlite3/secure/src/sqlite3.c:8135:22: note: previous declaration of 'i64' was here
sqlite3/secure/src/extensionfunctions.c:402:15: error: static declaration of 'acosh' follows non-static declaration
sqlite3/secure/src/extensionfunctions.c:410:15: error: static declaration of 'asinh' follows non-static declaration
sqlite3/secure/src/extensionfunctions.c:418:15: error: static declaration of 'atanh' follows non-static declaration
sqlite3/secure/src/extensionfunctions.c:725:5: error: expected ')' before 'c'
sqlite3/secure/src/extensionfunctions.c:725:5: error: expected expression before ')' token
sqlite3/secure/src/extensionfunctions.c: In function 'print_elem':
sqlite3/secure/src/extensionfunctions.c:1966:3: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'int64_t' [-Wformat]
I removed -DSQLITE_ENABLE_EXTFUNC and all is well.

Thanks again.
utelle
Moderator
Moderator
Posts: 1125
Joined: Tue Jul 05, 2005 10:00 pm
Location: Cologne, Germany
Contact:

Re: wxSQLite3, sqlite3, static library and encryption

Post by utelle »

paulcrown wrote: The error I had using -DSQLITE_ENABLE_EXTFUNC with the compile is:

Code: Select all

In file included from sqlite3/secure/src/sqlite3secure.c:58:0:
sqlite3/secure/src/extensionfunctions.c:213:25: error: conflicting types for 'i64'
sqlite3/secure/src/sqlite3.c:8135:22: note: previous declaration of 'i64' was here
sqlite3/secure/src/extensionfunctions.c:402:15: error: static declaration of 'acosh' follows non-static declaration
sqlite3/secure/src/extensionfunctions.c:410:15: error: static declaration of 'asinh' follows non-static declaration
sqlite3/secure/src/extensionfunctions.c:418:15: error: static declaration of 'atanh' follows non-static declaration
sqlite3/secure/src/extensionfunctions.c:725:5: error: expected ')' before 'c'
sqlite3/secure/src/extensionfunctions.c:725:5: error: expected expression before ')' token
sqlite3/secure/src/extensionfunctions.c: In function 'print_elem':
sqlite3/secure/src/extensionfunctions.c:1966:3: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'int64_t' [-Wformat]
I removed -DSQLITE_ENABLE_EXTFUNC and all is well.
Some minor modifications to extensionfunctions.c are required to make it compile in your case.

Remove comment characters in line 112 ff to solve errors in line 402, 410, 418 and probably 725:

Code: Select all

#define HAVE_ACOSH 1
#define HAVE_ASINH 1
#define HAVE_ATANH 1
#define HAVE_SINH 1
#define HAVE_COSH 1
#define HAVE_TANH 1
#define HAVE_LOG10 1
#define HAVE_ISBLANK 1
Adding comment characters in line 213 probably solves error related to i64 type:

Code: Select all

//typedef int64_t         i64;
I haven't tested it myself, though. If your code doesn't make use of the extension functions, then of course removing -DSQLITE_ENABLE_EXTFUNC is the quickest solution.

Regards,

Ulrich
Post Reply