diff --git a/README.md b/README.md index 474b29a..4d2ca85 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # SnowflakeService -A tool/microservice to centrally generate snowflake IDs. Can be run distributed. +A tool/microservice to centrally generate snowflake IDs. ![GitHub License](https://img.shields.io/github/license/B00tLoad/SnowflakeService) ![GitHub Issues or Pull Requests](https://img.shields.io/github/issues/B00tLoad/SnowflakeService) @@ -12,7 +12,8 @@ This utility uses: - The format and name of [Twitter's Snowflake IDs](https://blog.twitter.com/engineering/en_us/a/2010/announcing-snowflake) - [Javalin](https://github.com/javalin/javalin) - [QOS.CH's Logback](https://github.com/qos-ch/logback) -- [Square's OkHttp3](https://github.com/square/okhttp) +- [Google's gson](https://github.com/google/gson) +- [hyperxpro's Brotli4j](https://github.com/hyperxpro/Brotli4j) - [B00tLoad_'s ConfigurationUtilities](https://github.com/B00tLoad/Configurationutilities) ## Installation @@ -20,17 +21,17 @@ This utility uses: ### Docker (Recommended) ```bash - docker pull bootmediaalix/snowflake-service - docker run bootmediaalix/snowflake-service -e %{set required .env, see below} -p 95674:95674 -v /data/b00tload-services/snowflake:%desired path on host% + docker pull bootmediaalix/snowflakeservice + docker run bootmediaalix/snowflakeservice -e %{set required .env, see below} -p 95674:95674 -v /data/b00tload-services/snowflake:%desired path on host% ``` ### Containerless A containerless installation is possible, although not supported. For development convenience the application base directory is located in `~/.b00tload-services/snowflake` instead of `/data/b00tload-services/snowflake`. If you want to work containerless you are on your own. ## Environment Variables -To run this project, you will need to add the following environment variables to your .env file +To run this project, you may add the following environment variables to your .env file -`EPOCH` - the starting time of the snowflake +`EPOCH` - the starting time of the snowflake (defaults to 01.01.2024 12:00 AM) `MACHINE_ID_BITS` - the amount of bits used for the machine ID @@ -38,20 +39,6 @@ To run this project, you will need to add the following environment variables to `MACHINE_ID` - the ID of the generator -**_or_** - -`ORCHESTRATOR_IP` - an IP where a snowflake orchestrator is available to fetch all above values - -**_or_** - -`EPOCH` - the starting time of the snowflake - -`MACHINE_ID_BITS` - the amount of bits used for the machine ID - -`SEQUENCE_BITS` - the amount of bits used for the sequence counter - -with neither `MACHINE_ID` nor `ORCHESTRATOR_IP` set, the Service will start as an orchestrator itself with MACHINE_ID = 0. - ## API Reference @@ -59,64 +46,16 @@ with neither `MACHINE_ID` nor `ORCHESTRATOR_IP` set, the Service will start as a #### Get an ID ```http - GET /flake + GET /generate ``` Response example: ```json { - "snowflake": "" + "id": "50990430426234880" } ``` -#### Register worker (internal) - -```http request -POST /orchestra/register -``` - -| Header | Description | -|:-------|:----------------------------------------------------------------------------------| -| `name` | **Required**. A name to recognize the worker by (e.g. hostname+pid+random number) | - -Response example: -Http Status: 200 -```json -{ - "machineBits": "", - "workerid": "", - "sequenceBits": "", - "epoch": "" -} -``` -Response on error: -If the version of the client and the orchestrator do not match the orchestrator will respond with a HTTP code 426. The required version will be - -| Value name | Value description | -|:-------------------------|:--------------------------| -| Http Status | 426 - Upgrade required | -| `Upgrade` Header content | required software version | - -| Value name | Value description | -|:-------------------------|:--------------------------| -| Http Status | 409 - Conflict | -| `Upgrade` Header content | required software version | -#### Worker heartbeat (internal) -```http request -GET /orchestra/heartbeat -``` -| Parameter | Description | -|:------------|:------------------------------------------| -| `name` | the ID used in registration | -| `workerID` | the workerID assigned by the orchestrator | - -HTTP status code responses: - -| Status | Description | -|:-------|:----------------------------| -| `200` | Ok | -| `409` | Conflict, please reregister | - ## Maintainer - [@B00tLoad_](https://www.github.com/B00tLoad) @@ -129,5 +68,5 @@ HTTP status code responses: ## Support -For support, open a ticket or email me at alix (at) ja-lol-ey (dot) de. +For support, [open a ticket](https://github.com/B00tLoad/SnowflakeService/issues) or email me at alix (at) ja-lol-ey (dot) de. diff --git a/pom.xml b/pom.xml index d1a36da..0965ec5 100644 --- a/pom.xml +++ b/pom.xml @@ -6,10 +6,10 @@ space.b00tload.services SnowflakeService - 1.0-SNAPSHOT + 1.0.0-rc1 SnowflakeService - A tool/microservice to centrally generate snowflake IDs. Can be run distributed. + A tool/microservice to centrally generate snowflake IDs. 2024 https://github.com/B00tLoad/SnowflakeService @@ -65,18 +65,28 @@ ch.qos.logback logback-classic - 1.5.5 + 1.5.6 org.jetbrains annotations - 24.0.1 + 24.1.0 compile - com.squareup.okhttp3 - okhttp - 4.12.0 + io.javalin + javalin + 6.1.3 + + + com.aayushatharva.brotli4j + brotli4j + 1.16.0 + + + com.google.code.gson + gson + 2.10.1 @@ -148,7 +158,7 @@ - ${artifactId} + ${project.artifactId} jar-with-dependencies diff --git a/src/main/java/space/b00tload/services/snowflake/ConfigurationValues.java b/src/main/java/space/b00tload/services/snowflake/ConfigurationValues.java index 42909f3..2896d1e 100644 --- a/src/main/java/space/b00tload/services/snowflake/ConfigurationValues.java +++ b/src/main/java/space/b00tload/services/snowflake/ConfigurationValues.java @@ -2,12 +2,29 @@ package space.b00tload.services.snowflake; import space.b00tload.utils.configuration.ConfigValues; +/** + * All values to be configures + * @author Alix von Schirp + * @since 1.0.0 + * @version 1.0.0 + */ public enum ConfigurationValues implements ConfigValues { + /** + * Bits reserved for machine id + */ MACHINE_ID_BITS("machinebits", "M", "MACHINE_ID_BITS", "bitcount.machine", "10"), + /** + * Bits reserved for sequence + */ SEQUENCE_BITS("sequencebits", "S", "SEQUENCE_BITS", "bitcount.sequence", "12"), + /** + * Selected epoch + */ EPOCH("epoch", "E", "EPOCH", "epoch", "1704067200000"), - MACHINE_ID("machineid", "I", "MACHINE_ID", "machineid", "-1"), - ORCHESTRATOR_IP("orchestrator", "O", "ORCHESTRATOR_IP", "orchestrator.ip", "http://disabled"), + /** + * this machines id + */ + MACHINE_ID("machineid", "I", "MACHINE_ID", "machineid", "0"), ; private final String cliFlag; diff --git a/src/main/java/space/b00tload/services/snowflake/SnowflakeIDGenerator.java b/src/main/java/space/b00tload/services/snowflake/SnowflakeIDGenerator.java index 7627b06..646642c 100644 --- a/src/main/java/space/b00tload/services/snowflake/SnowflakeIDGenerator.java +++ b/src/main/java/space/b00tload/services/snowflake/SnowflakeIDGenerator.java @@ -20,6 +20,9 @@ import static java.lang.System.currentTimeMillis; public class SnowflakeIDGenerator { //Singleton instance + /** + * Singleton instance + */ private static SnowflakeIDGenerator INSTANCE; // Constants @@ -84,10 +87,10 @@ public class SnowflakeIDGenerator { /** * Used for initiating constants and state and performing bound checks on {@code EPOCH}, {@code MAX_TIMESTAMP} and {@code machineId}. * - * @param machineId The machineID used for generation // expected to be null if not initialized by orchestrator, loaded from config - * @param epoch The snowflake epoch used for generation // expected to be null if not initialized by orchestrator, loaded from config - * @param machineIdBits The amount of bits used for the machineID // expected to be null if not initialized by orchestrator, loaded from config - * @param sequenceBits The amount of bits used for sequence counter // expected to be null if not initialized by orchestrator, loaded from config + * @param machineId The machineID used for generation // expected to be null, loaded from config + * @param epoch The snowflake epoch used for generation // expected to be null, loaded from config + * @param machineIdBits The amount of bits used for the machineID // expected to be null, loaded from config + * @param sequenceBits The amount of bits used for sequence counter // expected to be null, loaded from config * @throws IllegalArgumentException if any bound checks fail. */ private SnowflakeIDGenerator(@Nullable Long machineId, @Nullable Long epoch, @Nullable Long machineIdBits, @Nullable Long sequenceBits) throws IllegalArgumentException{ @@ -129,6 +132,7 @@ public class SnowflakeIDGenerator { * Generates a new snowflake ID.
* It uses a long (64-bit signed int) made up of (in default configuration): * + * * * * @@ -152,7 +156,7 @@ public class SnowflakeIDGenerator { *
Bit layout
bit(s)content
* Generation will be halted * * @return the generated ID @@ -186,6 +190,13 @@ public class SnowflakeIDGenerator { // Wait for the next timestamp + + /** + * Blocks thread until next timestemp (=next milli) + * + * @param currentTimestamp the current milli + * @return the awaited milli + */ private long waitForNextTimestamp(long currentTimestamp) { long nextTimestamp = currentTimeMillis(); while (nextTimestamp <= currentTimestamp) { @@ -194,58 +205,19 @@ public class SnowflakeIDGenerator { return nextTimestamp; } - public static void init(long machineId, long epoch, long machineIdBits, long sequenceBits){ - INSTANCE = new SnowflakeIDGenerator(machineId, epoch, machineIdBits, sequenceBits); - } - public static void init(){ INSTANCE = new SnowflakeIDGenerator(null, null, null, null); } + /** + * Getter for the singleton instance. Inits singleton if needed. + * @return the singleton instance. + */ public static SnowflakeIDGenerator getInstance() { - if(Objects.isNull(INSTANCE)) throw new UnsupportedOperationException("SnowflakeIDGenerator is not initialized."); +// if(Objects.isNull(INSTANCE)) throw new UnsupportedOperationException("SnowflakeIDGenerator is not initialized."); + if(Objects.isNull(INSTANCE)) init(); return INSTANCE; } - - public long getEPOCH() { - return EPOCH; - } - - public long getMACHINE_ID_BITS() { - return MACHINE_ID_BITS; - } - - public long getSEQUENCE_BITS() { - return SEQUENCE_BITS; - } - - public long getMAX_MACHINE_ID() { - return MAX_MACHINE_ID; - } - - public long getMAX_SEQUENCE() { - return MAX_SEQUENCE; - } - - public long getMACHINE_ID_SHIFT() { - return MACHINE_ID_SHIFT; - } - - public long getTIMESTAMP_SHIFT() { - return TIMESTAMP_SHIFT; - } - - public long getMachineId() { - return machineId; - } - - public long getTIMESTAMP_BITS() { - return TIMESTAMP_BITS; - } - - public long getMAX_TIMESTAMP() { - return MAX_TIMESTAMP; - } } diff --git a/src/main/java/space/b00tload/services/snowflake/SnowflakeService.java b/src/main/java/space/b00tload/services/snowflake/SnowflakeService.java index 9d4fbdd..8b8e6c9 100644 --- a/src/main/java/space/b00tload/services/snowflake/SnowflakeService.java +++ b/src/main/java/space/b00tload/services/snowflake/SnowflakeService.java @@ -4,31 +4,45 @@ import ch.qos.logback.classic.LoggerContext; import ch.qos.logback.classic.joran.JoranConfigurator; import ch.qos.logback.core.joran.spi.JoranException; import ch.qos.logback.core.util.StatusPrinter2; -import okhttp3.*; +import com.google.gson.JsonObject; +import io.javalin.Javalin; +import io.javalin.http.ContentType; +import io.javalin.http.HandlerType; +import io.javalin.router.Endpoint; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import space.b00tload.utils.configuration.Configuration; -import java.io.IOException; import java.nio.file.Paths; import java.util.List; import java.util.Objects; -import java.util.Random; +/** + * Main class orchestrating between generator, webserver and config + * + * @author Alix von Schirp + * @version 1.0.0 + * @since 1.0.0 + */ public class SnowflakeService { + /** + * This software's version + */ private static String SOFTWARE_VERSION; + /** + * Base dir for config and tmp data + */ private static String APPLICATION_BASE; - private static String USER_AGENT; - private static Logger LOGGER; - private static String INSTANCE_NAME; + /** + * Initializes tool, generator and webserver. + * @param args cli args + */ public static void main(String[] args) { //Set up constants SOFTWARE_VERSION = Objects.requireNonNullElse(SnowflakeService.class.getPackage().getImplementationVersion(), "0.0.1-indev"); - USER_AGENT = "SnowflakeService " + SOFTWARE_VERSION + "(" + System.getProperty("os.name") + "; " + System.getProperty("os.arch") + ") Java/" + System.getProperty("java.version"); APPLICATION_BASE = List.of(args).contains("--docker") ? Paths.get("data", "b00tload-tools", "snowflake").toString() : Paths.get(System.getProperty("user.home"), ".b00tload-tools", "snowflake").toString(); - INSTANCE_NAME = getHostname() + "_" + getPid() + "_" + new Random().nextInt(9999); //Set up logger LoggerContext loggerContext = (LoggerContext) LoggerFactory.getILoggerFactory(); @@ -45,52 +59,25 @@ public class SnowflakeService { (new StatusPrinter2()).printInCaseOfErrorsOrWarnings(loggerContext); //Set up logger - LOGGER = LoggerFactory.getLogger(SnowflakeService.class); + Logger LOGGER = LoggerFactory.getLogger(SnowflakeService.class); //Init config Configuration.init(args, SOFTWARE_VERSION, APPLICATION_BASE, ConfigurationValues.values()); - long machineID; - long sequenceBits; - long machineBits; - long epoch; - - if(!Configuration.getInstance().get(ConfigurationValues.ORCHESTRATOR_IP).equals(ConfigurationValues.ORCHESTRATOR_IP.getDefaultValue())){ - OkHttpClient httpClient = new OkHttpClient(); - try { - try (Response r = httpClient.newCall( - new Request.Builder() - .url(Configuration.getInstance().get(ConfigurationValues.ORCHESTRATOR_IP)) - .post( - new FormBody.Builder().addEncoded("name", INSTANCE_NAME).build() - ) - .build() - ).execute()){ - if(r.code() == 200){ - - } - } - } catch (IOException e) { - throw new RuntimeException(e); - } - } else if((Long.parseLong(Configuration.getInstance().get(ConfigurationValues.MACHINE_ID))) != -1L) { - - } else { - - } + //Init webserver + Javalin endpointServer = Javalin.create(config -> { + config.http.brotliAndGzipCompression(); + config.http.prefer405over404 = true; + config.requestLogger.http((ctx, executionTimeMs) -> { + LoggerFactory.getLogger(config.getClass()).info("{} served in {}ms to {}(UA: \"{}\")", ctx.fullUrl(), executionTimeMs, ctx.req().getRemoteAddr(), ctx.userAgent()); + }); + }); + endpointServer.addEndpoint(new Endpoint(HandlerType.GET, "generate", ctx -> { + JsonObject ret = new JsonObject(); + ret.addProperty("id", SnowflakeIDGenerator.getInstance().generateID()); + ctx.status(200).result(ret.toString()).contentType(ContentType.APPLICATION_JSON); + })); + endpointServer.start(95674); } - - private static String getHostname(){ - String hostname = System.getenv("COMPUTERNAME"); // On Windows - if (hostname == null || hostname.isEmpty()) { - hostname = System.getenv("HOSTNAME"); // On Unix/Linux - } - return hostname; - } - - private static long getPid(){ - return ProcessHandle.current().pid(); - } - } diff --git a/src/main/resources/config/logback/logback-bare.xml b/src/main/resources/config/logback/logback-bare.xml index 9f5534d..089e742 100644 --- a/src/main/resources/config/logback/logback-bare.xml +++ b/src/main/resources/config/logback/logback-bare.xml @@ -14,6 +14,9 @@ ERROR DENY + + INFO + @@ -26,16 +29,45 @@ ${LOG_PATH}/snowflake.log - ${LOG_ARCHIVE}/rollingfile.log%d{yyy-MM-dd}.lol.gz + ${LOG_ARCHIVE}/snowflake.rolling.%d{yyy-MM-dd}.log.gz + + INFO + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n + + + + ${LOG_PATH}/snowflake.debug.log + + ${LOG_ARCHIVE}/snowflake.debug.rolling.%d{yyy-MM-dd}.log.gz + + + DEBUG + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n + + + + ${LOG_PATH}/snowflake.trace.log + + ${LOG_ARCHIVE}/snowflake.trace.rolling.%d{yyy-MM-dd}.log.gz + + + TRACE + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n - + + + \ No newline at end of file diff --git a/src/main/resources/config/logback/logback-docker.xml b/src/main/resources/config/logback/logback-docker.xml index f6cf91c..7f074d7 100644 --- a/src/main/resources/config/logback/logback-docker.xml +++ b/src/main/resources/config/logback/logback-docker.xml @@ -14,6 +14,9 @@ ERROR DENY + + INFO + @@ -26,16 +29,45 @@ ${LOG_PATH}/snowflake.log - ${LOG_ARCHIVE}/rollingfile.log%d{yyy-MM-dd}.lol.gz + ${LOG_ARCHIVE}/snowflake.rolling.%d{yyy-MM-dd}.log.gz + + INFO + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n + + + + ${LOG_PATH}/snowflake.debug.log + + ${LOG_ARCHIVE}/snowflake.debug.rolling.%d{yyy-MM-dd}.log.gz + + + DEBUG + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n + + + + ${LOG_PATH}/snowflake.trace.log + + ${LOG_ARCHIVE}/snowflake.trace.rolling.%d{yyy-MM-dd}.log.gz + + + TRACE + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n - + + + \ No newline at end of file