(gen/sample (s/gen :mega-corp/insurance-policy))
Use case: How to define mass data structure using spec and to get the data from that using “conform”.
Need sample use case and accustomed project with that.
Require more details on why we need this spec and what is the business advantage?
Use case: To mock test data for resources on validating the specs as a whole data structure. (A sample project or reference available on internet is also required)
How to get the project built using Jenkins or any hosting tools. We would have a demo to do that with pipeline or docker plugin.
How to call web services from clojure & to hop around the different environments? How are we going to handle/pass the security configuration in between the layers?
We use “:gen-class” for all the namespace definition. Is this mandatory for running the application as uber jar? What is the purpose of including this :gen-class?
More examples (different usage scenarios) and elaboration required for Macros.
Use case: Require to know Clojure with Hadoop interaction for data enrichment with clojure specs.
Does clojure provide any utility to attach a file and send an email? Configuring logs on masking the sensitive data.
How to manage real time dependency management between projects
Any industry design patterns for clojure?
What is Profile.clj capable of doing? Is it like getting the jar from repo, defining plugins, is that all? Or do we have more to explore. (need more clarity and reference links)
What is Project.clj capable of doing to the project level and where does this scope live? Unlike the above one, require reference materials.
Insurance company "Megacorp" offers three pet insurance policies:
Can spec help us process applications for these policies?
See examples/parsing-with-spec
(valid?) expound
conformWhen there are multiple options to match against
It is a guiding principle of Clojure to represent information as data (not objects)
Important properties of Clojure systems are represented and conveyed by the shape of the data
runtime types are maps, vectors, sets
shape is not captured or checked anywhere
Manual parsing and error reporting is not good enough
Defining specifications is optional
Validation
Error reporting
Destructuring
Instrumentation
Test-data generation
Generative test generation
It is new
Specs are code
however code is data and you can create specs on the fly
You can do without it
however manual approaches become difficult as complexity rises
Specifying the shape of inputs is hard
good error messages are hard for complex requirements
coming up with test data can be tedious
testing often involves reimplementation
Additional tooling
Expound (nice error messages)
Data generators
test.check
Implement validation quickly
Implement parsing quickly
Deliver correct, well tested solutions
Use the same technology in code and data interfaces
learn once, apply in many contexts
see examples/parsing-with-spec/test/parsing-with-spec/generated-test
(gen/sample (s/gen :mega-corp/insurance-policy))
(stest/check `my-function)
Building projects with Jenkins
See examples/jenkins-build-server
How Clojure projects are built and executed locally
How to set up a build server to automate test/build/deploy
lein run
lein uberjar
java -jar myapp.jar
lein ring uberjar
lein install
lein deploy
Easy to execute
May need to pull dependencies
Your application and all dependencies in a single JAR file
Easy to deploy
Easy to execute
Versioned
Preservable
Executes an uberjar
Define a main entrypoint in project.clj
:main myapp.core
src/myapp/core.clj
(ns myapp.core) (defn -main [& args] ...)
Alternatively, specify an entrypoint from the commandline
java -jar myapp.jar -m myapp.core/-main
Sets up a main entry point to start the webservice
Equivalent to
(ns myapp.core)
(defn -main [& args]
(run-jetty handler {:port 3000}))Builds an uberjar and puts it in your local Maven repository ~/.m2
Useful for testing library snapshots and building from source
Does not publish your artifact
Publishes your artifact to a repository
Repositories can be
public (Clojars, Maven Central)
private (Hosted/S3/Self managed)
https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md
Easy… if you have a repository…
Not the same as deploying your application!
Amazon S3 is a low-maintenance choice:
Artifactory/Nexus/Archiva
Deps: https://www.deps.co/
Remeber to include a repositories section in your project.clj
To publish to a private repository
To pull dependencies from a private repository
:repositories [["private" {:url "s3p://mybucket/releases/" :no-auth true}]]Build an uberjar (or Docker container)
Get the artifact to the host server
Run it
Tomcat? Drop a WAR in a folder
AWS Elastic Beanstalk? Roll out a new docker container
AWS Lambda? Upload a new JAR
Kubernetes? Roll out a new docker container
Heroku? Deploy from git
Install Jenkins
Install Leiningen
Add a build
Virtualization
Dockerfile specifies a parent and setup tasks
Building starts with an image and runs the setup tasks, creates a container
Run a container and it behaves like a stand alone computer
docker build -t <image-name> . to create a container from a Dockerfile
docker run <image-name> to run it
docker ps to see running containers
docker exec -ti <container-id> bash to get a shell in a running container
docker stop <container-id> to stop it
Install docker
Create a Dockerfile to extend the base Jenkins image
see examples/jenkins-build-server/Dockerfile
add steps to install Leiningen
Create a Makefile or similar to automate tasks
see examples/jenkins-build-server/Makefile
make run to start the server
Open the UI: http://localhost:8080/
Enter password from console log
Install suggested plugins
Create admin user
An uberjar
Docker image
Choose Freestyle project
Configure source code management
parsing-with-spec is a subdirectory of the enterprise-clojure repository
sparse checkout path only gets a subdirectory
Under "Build", add build steps, shell command
cd examples/parsing-with-spec && lein test
cd examples/parsing-with-spec && lein install
or docker build
don’t need to change directory if project is in root
Save
Build now
Check the console logs
tests passed
jar created
java -jar myapp.jar
Use environment variables to behave differently
Can extend a prebuilt docker image (or roll your own)
Set up jobs for each project
If the build succeeds, deploy
lein new compojure customer-data-service
See routes
Start: lein ring server-headless
Test: curl http://localhost:3000/
Change code
curl again
Ring is a library that abstracts the details of HTTP into a unified API
modular components
Handlers - functions that take requests and return responses
Request - map of data about the request (params, body, etc)
Response - map containing status, headers, body
Middleware - the mechanism for modular components, higher order functions
Middleware are functions that return functions:
(defn wrap-user [handler]
(fn [request]
(if-let [user-id (-> request :session :user-id)]
(let [user (get-user-by-id user-id)]
(handler (assoc request :user user)))
(handler request))))Takes a handler as input
The function returned
can modify the request before passing it to the handler
can modify the result from the handler before returning it
is itself a handler
Takes a handler and creates an augmented handler.
Convenient for composing handlers together:
(def app
(-> handler
(wrap-user)
(wrap-content-type "text/html")
(wrap-keyword-params)
(wrap-params)))Middleware pattern
A routing library
/insurance-policy/corgi-cover is a route
/insurance-policy is fixed root
/corgi-cover is a policy ID
could be poodle-protection or poodle-protection-platinum
(GET "/insurance-policy/:id" [id] (fetch-policy id))
Public API creation: https://github.com/metosin/compojure-api
Public API creation: https://clojure-liberator.github.io/liberator/
clj-http https://github.com/dakrone/clj-http
See examples/communicating-services/insurance-policy-application-processor
[clj-http "0.6.0"]
(ns myns (:require [clj-http.client :as client])) (client/get (str "http://customer-data-service:3000/customer/" id))
See examples/communicating-services
See examples/communicating-services/docker-compose.yml
Building
Deploying
Reloading code while developing multiple services
Follow the Twelve-Factor App https://12factor.net/
Store configuration in the environment
Don’t store configuration in source control
Don’t expose endpoints publicly
Authentication/authorization for public endpoints: https://github.com/cemerick/friend
:gen-classWhat is the purpose of :gen-class?
gen-class creates a Java class
You can specify AOT namespaces in the project.clj file
lein compile to build the class
Prefer using reify, deftype, defrecord to implement Java classes
Avoid gen-class and AOT
But why is it used so often?
An uberjar contains all your project dependencies.
If it contains a main class then it will also be executable:
java -jar myuber.jar
A common way to make an executable uberjar is:
src/myns/core.clj
(ns myns.core (:gen-class)) (defn -main [& args] (println "Hello World"))
project.clj
:main myns.core :aot [myns.core]
(ns my.app) (def password (System/getenv "PASSWORD"))
Behaves differently if evaluated during AOT than during Runtime.
In AOT it is captured during the build prior to deployment.
At Runtime it is whatever is in the environment when the namespace is loaded.
Shipping a binary without the source code
Marginally speeding up start time
Generating classes loadable directly from Java for interop purposes (Hadoop)
Platforms such as Android do not support custom class loaders for running new bytecode at runtime.
:gen-class mandatory?No
See examples/aot/too-much-aot
An example where aot is causing undesirable behavior
See examples/aot/no-aot
Clojure provides an entrypoint java -cp myuber.jar clojure.main -m myns.core
Clojure compiles all code you load on-the-fly into JVM bytecode
See examples/aot/little-aot
Avoid transitive aot by providing a bootstrap
Produces an executable jar
Examples (different usage scenarios) and elaboration required for Macros.
Many Clojure primitives such as defn are macros
Macros provide syntax but are not values
https://lispcast.com/when-to-use-a-macro/
rarely
compile time
unevaluated arguments
inline code
core.async
macros over state machines to implement Communicating Sequential Processes (CSP)
provides a new syntax (go blocks)
available as a library
Require to know Clojure with Hadoop interaction for data enrichment with clojure specs.
(send-message
{:host "mail.isp.net"}
{:from "me@draines.com"
:to "foo@example.com"
:subject "Hi!"
:body [{:type "text/html"
:content "<b>Test!</b>"}
{:type :attachment
:content (java.io.File. "/tmp/foo.txt")}
{:type :inline
:content (java.io.File. "/tmp/a.pdf")
:content-type "application/pdf"}]})You need a mail server. (AWS Simple Email Service works well https://aws.amazon.com/ses/)
Clojure.tools.logging https://github.com/clojure/tools.logging
log4j
logback
commons logging
slf4j
Standard Java style configuration
"Bear Traps: the Logging Edition" https://www.youtube.com/watch?v=jjuj-D4-hCE https://github.com/AlexanderMann/unclogging
How to manage real time dependency management between projects
lein checkouts
lein install
What is Profile.clj capable of doing?
profile.clj is merged with any project.clj.
Having a profile.clj allows you to have common dependencies or tasks.
Example: lein-test-refresh plugin
Is it like getting the jar from repo, defining plugins, is that all?
Pretty much…
Capable of anything project.clj can do, but usually just for tools.
Or do we have more to explore. (need more clarity and reference links)
lein-test-refresh
Ancient
Ultra
humane-test-output
Eastwood
Kibit
Bikeshed
Alembic
What is Project.clj capable of doing to the project? Where does this scope live? Unlike the above one, require reference materials.