Subscribe to forum Forum

Hi,

Is there a way to automatically clean up log files that were created by setting the connection to rotate?

Hello Cesar,

This functionality is currently not built-in, but already on our feature request list.

You could also achieve the same functionality with a simple background thread which periodically checks if there are too many log files and then deletes unnecessary logs. We have an example (for Delphi) which does exactly this. If you are interested, I can send you the example snippet via email.

Yes, I would be interested in the code snippet. Thanks.

Sent! Please let me know if you have any questions about the example.

Hello,
Can I ask for the cleanup code too?

Hello Piotr,

Sure. Since several other people have also asked for this, I will post the unit right here.

unit LogCleanup;

interface

uses
  SiAuto, SmartInspect, SysUtils, Classes, SyncObjs;

const
  CFileName = 'log.sil';
  CMaxFiles = 10;
  CCleanupInterval = 1000 * 60 * 60;

type
  TLogCleanupThread = class(TThread)
  private
    FMaxFiles: Integer;
    FEvent: TEvent;
    FFileName: String;
    FCleanupInterval: Integer;
    procedure Cleanup;
  protected
    procedure Execute; override;
  public
    constructor Create(const AFileName: String = CFileName;
      const AMaxFiles: Integer = CMaxFiles;
      const ACleanupInterval: Integer = CCleanupInterval);
    destructor Destroy; override;
    procedure Cancel;
    property MaxFiles: Integer read FMaxFiles;
    property CleanupInterval: Integer read FCleanupInterval;
  end;

implementation

var
  GSession: TSiSession;

{ TLogCleanupThread }

procedure TLogCleanupThread.Cancel;
begin
  GSession.EnterMethod(Self, 'Cancel');
  GSession.LogWarning('Cancelling!');
  Terminate;
  FEvent.SetEvent;
  GSession.LeaveMethod(Self, 'Cancel');
end;

procedure TLogCleanupThread.Cleanup;
var
  LSearchRec: TSearchRec;
  LErrorCode: Integer;
  LMask, LPath: String;
  LLogFile: String;
  LFiles: TStringList;
  I: Integer;
begin
  GSession.EnterMethod(Self, 'Cleanup');

  LMask := ChangeFileExt(FFileName, '') + '*' + ExtractFileExt(FFileName);
  LErrorCode := FindFirst(LMask, 0, LSearchRec);

  if LErrorCode = 0 then
  begin
    try
      LFiles := TStringList.Create;
      try
        LFiles.Sorted := True;
        LPath := IncludeTrailingPathDelimiter(ExtractFilePath(FFileName));

        repeat
          LLogFile := LPath + LSearchRec.Name;
          LFiles.Add(LLogFile);
        until FindNext(LSearchRec) <> 0;

        GSession.LogStringList('LFiles', LFiles);

        if LFiles.Count > FMaxFiles then
        begin
          for I := 0 to Pred(LFiles.Count - FMaxFiles) do
          begin
            GSession.LogMessage(LFiles[I]);
            if not DeleteFile(LFiles[I]) then
            begin
              GSession.LogLastError('DeleteFile');
            end;
          end;
        end;
      finally
        LFiles.Free;
      end;
    finally
      FindClose(LSearchRec);
    end;
  end else
  begin
    GSession.LogString('LMask', LMask);
    GSession.LogWin32Error('FindFirst', LErrorCode);
  end;

  GSession.LeaveMethod(Self, 'Cleanup');
end;

constructor TLogCleanupThread.Create(const AFileName: String;
  const AMaxFiles: Integer; const ACleanupInterval: Integer);
begin
  inherited Create(True);
  FFileName := AFileName;
  FMaxFiles := AMaxFiles;
  FCleanupInterval := ACleanupInterval;
  FEvent := TEvent.Create(nil, False, False, '');
end;

destructor TLogCleanupThread.Destroy;
begin
  FreeAndNil(FEvent);
  inherited;
end;

procedure TLogCleanupThread.Execute;
begin
  GSession.EnterThread('Log Cleanup');
  while not Terminated do
  begin
    Cleanup;
    GSession.LogMessage('Waiting..');
    FEvent.WaitFor(FCleanupInterval);
  end;
  GSession.LeaveThread('Log Cleanup');
end;

initialization
  GSession := Si.AddSession('Log Cleanup');
end.

This unit can be used as follows:

type
  TForm1 = class(TForm)
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
  private
    FLogCleanup: TLogCleanupThread;
  end;

...

procedure TForm1.FormCreate(Sender: TObject);
begin
  FLogCleanup := TLogCleanupThread.Create('c:\Test\log.sil');
  FLogCleanup.Resume;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  FLogCleanup.Cancel;
  FLogCleanup.WaitFor;
  FreeAndNil(FLogCleanup);
end;

Most of the time the background worker thread is put to sleep and thus does not use any unnecessary resources. In a customizable interval, the background thread wakes up and checks if old log files should be deleted.

The filename parameter for TLogCleanupThread specifies the base name for the log files to delete. In the example above, the thread is responsible for deleting files with the mask "c:\Test\log*.sil". Moreover, the example uses the default settings for the delete interval (one hour) as well as the maximum allowed log files (10).

Hi Tobias,

I have just started using this code but found it unnecessary to have a background thread waiting for doing a cleanup. I just extracted the actual cleanup code and execute it at program startup. That's enough for my purpose and reduces the overhead.

Since I guess you will be adding this code to the smart inspect libraries, maybe you want to also offer this option.

twm

Hello,
I think that cleanup at startup is not enough for long time running server applications. After months or years of running, it may result in GBs of log files.

best regards
Piotr Rezmer

piotr rezmer wrote

I think that cleanup at startup is not enough for long time running server applications. After months or years of running, it may result in GBs of log files.

I agree. But many are not writing server apps but programs that get restarted at least once a day. Those might not want to have a background thread running all the time which does nothing. It's just a matter of having a choice.

twm

Hello all,

I think we will probably integrate the cleanup of old log files directly into the file protocol without using additional threads. On startup and whenever a new log file is created, the file protocol could then automatically delete unwanted/unneeded log files.

Hello,

Just to clarify this: we won't just add threads for new features to the libraries without a way to disable them. Some features like asynchronous logging require independent threads, but this is an optional feature (asynchronous logging is planned for SmartInspect 3.0). Just like twm said, we also believe that having a choice to disable such features is important.